Simplifying RAG for PDFs with File Search in the Responses API

While RAG can feel overwhelming, conducting searches across PDF files shouldn't be. A widely used approach involves parsing your PDFs, establishing chunking strategies, uploading those chunks to a storage provider, generating embeddings on the text chunks, and storing the embeddings in a vector database. And that's just the setup—incorporating retrieval into your LLM workflow requires even more steps. Enter file search—a hosted tool available in the Responses API. It lets you query your knowledge base and generate answers based on the retrieved content. In this cookbook, we'll upload PDFs to a vector store hosted by OpenAI and employ file search to retrieve additional context from this vector store for answering the questions created in the first step. We'll start by generating a small set of questions from PDFs extracted from OpenAI's blog (openai.com/news). *File search was previously available on the Assistants API. It's now available on the new Responses API, an API that can be stateful or stateless, and with new features like metadata filtering.* # Creating Vector Store with our PDFs