Mastering Raw Chain of Thought Management in GPT-OSS Models

GPT-OSS models expose a raw chain of thought (CoT) feature designed primarily for analysis and safety research by developers. This CoT is vital for tool calling performance since tools can integrate directly into the reasoning process. However, raw CoT may include sensitive or harmful content, or unintended disclosures like internal model instructions, so it's essential to never display it to end users. ## Harmony / chat template handling In GPT-OSS, the raw CoT is embedded within the harmony response format. If you're creating custom chat templates or processing tokens manually, consult the harmony guide beforehand. Key points to remember: - CoT appears in the `analysis` channel - Once a `final` channel message is generated in a later sampling step, discard all prior `analysis` messages. `commentary` channel function calls can persist - If the assistant's most recent message involves any tool call, retain `analysis` messages from before the last `final` until a new `final` is produced ## Chat Completions API There's no standard in the official OpenAI specs for CoT handling, as their hosted models don't support it yet. Adopt the OpenRouter convention instead, which includes: - Raw CoT appears in responses by default, unless `reasoning: { exclude: true }` is set in the request. More details [here](link-not-specified) - CoT is provided via a `reasoning` property on the output message - Delta events include a `reasoning` property for streaming - Future turns can ingest prior `reasoning` data, processing it per the chat template rules above Refer to the OpenRouter implementation for guidance when unsure. ## Responses API We've updated the Responses API spec to address CoT. Key additions: - A new `content` property on `reasoning` enables a user-friendly `summary` alongside raw CoT (keep raw hidden for users, ideal for research) - New content type: `reasoning_text` - New events: `response.reasoning_text.delta` for streaming raw CoT deltas and `response.reasoning_text.done` to signal CoT completion - Handle prior reasoning on follow-up turns as outlined in the chat template section **Item type changes** ``` type ReasoningItem = { id: string; type: "reasoning"; summary: SummaryContent[]; // new content: ReasoningTextContent[]; }; type ReasoningTextContent = { type: "reasoning_text"; text: string; }; type ReasoningTextDeltaEvent = { type: "response.reasoning_text.delta"; sequence_number: number; item_id: string; output_index: number; content_index: number; delta: string; }; type ReasoningTextDoneEvent = { type: "response.reasoning_text.done"; sequence_number: number; item_id: string; output_index: number; content_index: number; text: string; }; ``` **Event changes** ``` ... { type: "response.content_part.added" ... } { type: "response.reasoning_text.delta", sequence_number: 14, item_id: "rs_67f47a642e788191aec9b5c1a35ab3c3016f2c95937d6e91", output_index: 0, content_index: 0, delta: "The " } ... { type: "response.reasoning_text.done", sequence_number: 18, item_id: "rs_67f47a642e788191aec9b5c1a35ab3c3016f2c95937d6e91", output_index: 0, content_index: 0, text: "The user asked me to think" } ``` **Example responses output** ``` "output": [ { "type": "reasoning", "id": "rs_67f47a642e788191aec9b5c1a35ab3c3016f2c95937d6e91", "summary": [ { "type": "summary_text", "text": "**Calculating volume of gold for Pluto layer**\n\nStarting with the approximation..." } ], "content": [ { "type": "reasoning_text", "text": "The user asked me to think..." } ] } ] ``` ## Displaying raw CoT to end-users For chat interfaces, avoid exposing raw CoT to users due to risks of harmful content or leaks like developer instructions. Opt for a summarized version, akin to production APIs or ChatGPT, where a separate summarizer model filters and sanitizes content for safe display.