Imagine you're the AI lead at a consumer tech company with a grand vision: launching a single-entry-point digital voice assistant capable of assisting users with any request—whether it's managing their account, retrieving product details, or offering real-time advice.

Yet, bringing this vision to life is incredibly challenging. It demands developing and testing each use case via text interfaces first, integrating diverse tools and systems, and orchestrating them into a smooth experience. After reaching acceptable quality levels (which is hard to measure), you must refactor everything for voice interactions.

Thankfully, three recent OpenAI releases have dramatically simplified this process, offering tools to create and manage modular agentic workflows for voice with minimal setup:

• Responses API - an agentic API for easy engagement with our frontier models through managed stateful conversations, tracing of responses to enable evaluation, and built-in tools for file search, web search, computer use, and more

• Agents SDK - a lightweight, customizable open source framework for building and orchestrating workflows across many different agents, enabling your assistant to route inputs to the appropriate agent and to scale to support many use cases

• Voice agents - an extension of the Agents SDK to support the use of voice pipelines, enabling your agents to go from being text-base to being able to interpret and produce audio in just a few lines of code

This cookbook shows how to build a simple in-app voice assistant for a fictitious consumer application using the tools above. We'll create a Triage Agent that greets the user, determines their intent, and routes requests to one of three specialised agents:

• Search Agent - performs a web search via the built-in tooling of the Responses API to provide real-time information on the user's query

• Knowledge Agent - utilises the file search tooling of the Responses API to retrieve information from an OpenAI managed vector database

• Account Agent - uses function calling to provide the ability to trigger custom actions via API

Finally, we'll convert this workflow into a live voice assistant using the Agents SDK's Voice functionality, capturing microphone input, performing speech-to-text, routing through our agents, and responding with text-to-speech.