AcruxCore
Open-source LLMOps: version prompts, route LLM calls, trace and evaluate
AcruxCore is an open-source (Apache 2.0) LLMOps platform for engineering teams that build features and agents on large language models. Prompts live in one registry with version history, so a team can change and roll back a prompt without redeploying the app. An OpenAI-compatible gateway sits in front of many model providers and adds budgets, rate limits and caching. Every call is recorded as a trace with tokens, cost and latency, and a multi-step agent run becomes one trace. Evaluations score prompt versions against datasets with an LLM judge. SDKs for Python and TypeScript. Free to use, or self-host it with Docker.
Helicone
Self-host / Cloud
Self-host / Cloud Found in: Yigtwxx/Awesome-RAG-Production
Openllmetry
Open-source observability for your LLM application, based on OpenTelemetry 
Open-source observability for your LLM application, based on OpenTelemetry !GitHub Repo stars Found in: kyrolabs/awesome-langchain
Arize Phoenix
Open-source AI observability for LLM apps
Found in: Yigtwxx/Awesome-RAG-Production
LangSmith
LangSmith is an all-in-one platform to develop, debug, test, and monitor LLM applications, ensuring high performance, accuracy, and reliability.
LangSmith is an all-in-one platform to develop, debug, test, and monitor LLM applications, ensuring high performance, accuracy, and reliability.
Langfuse
An open-source LLM engineering platform for tracing, evaluation, prompt management, and metrics. [#opensource](https://github.com/langfuse/langfuse)
An open-source LLM engineering platform for tracing, evaluation, prompt management, and metrics. #opensource Found in: steven2358/awesome-generative-ai