Arize Phoenix
Open-source AI observability for LLM apps
About
Found in: Yigtwxx/Awesome-RAG-Production
Details
Arize Phoenix is an open-source AI observability platform designed for experimentation, evaluation, and troubleshooting of LLM applications. It provides OpenTelemetry-based tracing to capture runtime behavior, LLM-powered evaluation for benchmarking performance, versioned datasets for testing and fine-tuning, experiment tracking for prompt and model changes, a playground for optimizing prompts and comparing models, prompt management with version control and tagging, an AI engineering agent (PXI) for debugging and navigation, and a remote MCP server for connecting Claude Code, Cursor, and other MCP clients. The platform is vendor and language agnostic, with out-of-the-box support for popular frameworks like OpenAI Agents SDK, LangGraph, LlamaIndex, and providers such as OpenAI, Anthropic, and Google GenAI. It can run locally, in Jupyter notebooks, containerized, or in the cloud.