OpenAI Responses API Migration Guide: Replace Chat Completions and Assistants Safely

Last reviewed: August 30, 2026. API interfaces and product settings change; verify current official documentation before production deployment.

A Responses API migration is an application-architecture change, not a search-and-replace operation. Request fields, output items, conversation state, tool calls, streaming events, storage, and compatibility boundaries all need explicit tests before old endpoints are removed.

Inventory the current contract

List every endpoint, SDK version, model, message role, tool, response parser, storage assumption, retry rule, and downstream consumer used today. Record whether the application depends on Assistants threads, Chat Completions choices, function-call fields, JSON mode, or custom streaming parsers. Migration risk lives in these contracts, not in the number of lines changed.

Understand the Responses output model

Responses returns an ordered collection of output items rather than requiring all useful content to live in one message choice. Text, tool calls, reasoning-related items, and other events can have distinct shapes. Use the SDK's output_text helper only when plain text is truly the only required output. Production parsers should handle the documented item types and reject unexpected states safely.

Choose a conversation-state strategy

You can continue with a previous response identifier, use a conversation resource where supported, or manage history yourself. Each option affects storage, deletion, privacy, replay, and debugging. Do not mix state strategies accidentally. If your organization requires stateless or zero-retention behavior, confirm how reasoning items and encrypted content must be replayed under the current documentation.

Map instructions and messages

Separate stable application instructions from dynamic user input. Confirm how system or developer guidance is represented and whether previous instructions carry into the next call. Migration is an opportunity to remove duplicated prompts and move schemas into structured-output configuration rather than describing them repeatedly in prose.

Migrate tools one capability at a time

Start with one custom function and validate its schema, call identifier, arguments, execution result, and continuation request. Then add parallel calls, hosted tools, MCP, file search, or other features individually. Preserve the link between each tool call and its output. Never execute model-provided arguments without application-side authorization and validation.

const response = await client.responses.create({
  model: process.env.MODEL_ID,
  instructions: 'Answer concisely.',
  input: 'Give one migration checkpoint.',
});

console.log(response.output_text);
// For tools or structured data, parse documented output items instead.

Rewrite the streaming parser

Responses streaming uses typed server-sent events. Do not keep a parser that assumes every data line contains a Chat Completions delta. Handle creation, content deltas, tool events, completion, incomplete status, errors, and connection termination. Build fixtures from recorded non-sensitive events so parser tests do not require a live API.

Revisit retry and idempotency

A dropped connection does not always prove the upstream request did nothing. Separate transport retries from application retries, set timeouts, preserve request identifiers, and ensure tool side effects are idempotent. Background jobs need durable state and cancellation logic. A migration that increases automatic retries can create duplicate tool actions or unexpected cost.

Compare quality, latency and cost

Run representative evaluations against the old and new paths using the same business tasks. Measure task success, tool correctness, schema validity, time to first token, total latency, input and output tokens, cached tokens, and cost per successful task. A lower raw token count is not a win if recovery turns or validation failures increase.

Test gateway compatibility explicitly

An OpenAI-compatible gateway may support Chat Completions while only partially supporting Responses, hosted tools, background mode, conversation resources, or specific streaming events. Ask for documented support and run capability tests. Do not switch the Base URL in production based only on a successful plain-text response.

Use a staged cutover

Add a feature flag, mirror a safe sample when policy permits, canary a small percentage, compare outputs, and keep the old path available during the rollback window. Define acceptance thresholds before launch. After cutover, remove dead thread or message storage carefully and update runbooks, dashboards, privacy documentation, and on-call alerts.

A practical 30-minute diagnosis workflow

Begin by freezing changes and recording one failing request with its timestamp, safe endpoint, model, application-key fingerprint, status, structured error, request ID, latency, and retry count. Reproduce it with the smallest possible input and no optional tools. Compare the failing environment with one known-good environment, changing only one variable at a time: credential, Base URL, endpoint family, model, SDK version, streaming, then tool configuration. Check the provider status page when the failure appears suddenly across unrelated workloads. Do not rotate keys, switch models, change proxies, and increase retries simultaneously; that destroys the evidence needed to identify the cause. Once the minimal call works, add production features back individually and record which change reintroduces the failure.

Build a repeatable test matrix

Create automated tests for authentication, a short non-streaming response, a long streamed response, cancellation, structured output, one tool call, a controlled 4xx error, a simulated 5xx error, and a timeout. Run the matrix against every model and route the application officially supports. Store sanitized response fixtures so parsers can be tested without spending tokens or depending on a live service. Include a quality check, because a technically valid fallback can still fail the business task. Re-run the suite after SDK upgrades, model alias changes, editor updates, gateway changes, and provider deprecation notices. A dated capability matrix is more useful than a one-time claim that an endpoint is compatible.

Monitor the result after the fix

For Responses API migration, monitor request volume, success rate, error classes, retry amplification, time to first token, total latency, input and output usage, resolved model, route, and cost per successful task. Break dashboards down by environment and application key so one noisy client does not hide the rest. Alert on changes from the workload’s own baseline instead of choosing arbitrary global thresholds. Review the first hour and first day after a fix, then convert the diagnosis into a short runbook with owner, rollback step, and links to official documentation. Remove temporary debug logging once the evidence has been captured, especially if it could include prompts, file paths, or user data.

Prevent the same issue from returning

Move endpoint, model, timeout, and feature settings into reviewed configuration rather than scattering them through source code and individual laptops. Validate required variables at startup, reject unknown models, and expose a safe configuration summary that never includes secrets. Assign every application key and route an owner, environment, purpose, and rotation date. Subscribe to provider release and deprecation notices, but promote changes only after representative evaluations. Keep a tested rollback path and make emergency switches visible in logs and dashboards. Finally, review whether the original alert detected the customer impact early enough; if not, improve the signal while the incident evidence is still fresh.

Responses API migration: final production checklist

• Use the documented Base URL, credential type, endpoint and model ID.
• Start with a minimal reproducible request before enabling tools or agents.
• Classify errors before retrying and keep retries inside a total deadline.
• Log request IDs, route, model, latency and token usage without secrets.
• Test streaming, cancellation, failure recovery and rollback.
• Verify every gateway-specific feature instead of assuming complete compatibility.

Frequently asked questions

Official sources

• Migrate to Responses
• Responses API reference
• Conversation state
• Streaming responses

This article provides technical guidance, not a guarantee of compatibility, availability, pricing, or security certification.