A/B Split Test AI Agent Prompts: Supabase + OpenAI

This workflow enables A/B split testing of different AI agent prompts using Supabase for session management and OpenAI for response generation. Upon receiving a chat message, it checks a Supabase table for the session ID. If the session is new, it inserts the ID and randomly assigns either the baseline or alternative prompt (controlled via a configurable node). The assigned prompt is then used consistently for all messages in that session, ensuring reliable testing.

Ideal for optimizing LLM configurations in production, such as prompt variations or temperature settings, without disrupting user experience. In real estate lead management, it tests which prompt better qualifies leads, nurtures conversations, or books viewings, providing data-driven insights into agent effectiveness.

Benefits include easy setup with Supabase (create 'split_test_sessions' table), OpenAI integration for dynamic responses, and extensibility for metrics tracking or multi-variant tests. Save time on manual A/B experiments and scale agent improvements effortlessly.