Neura Market
Openllmetry logo

Openllmetry

Free

Open-source observability for your LLM application, based on OpenTelemetry ![GitHub Repo stars](https://img.shields.io/github/stars/traceloop/openllmetry?style=social)

Agent TracingFreeFree tier
Type
Open Source
Company
Traceloop
Visit Website GitHub (40 stars)

About Openllmetry

OpenLLMetry is an open-source observability tool for GenAI and LLM applications, built on top of OpenTelemetry. It provides complete tracing and monitoring by instrumenting LLM providers (OpenAI, Anthropic, Cohere, etc.), vector databases (Chroma, Pinecone, Qdrant, Weaviate, etc.), and frameworks (LangChain, LlamaIndex, Haystack, etc.) as well as standard OpenTelemetry data sources. It uses OpenTelemetry under the hood, allowing it to connect to existing observability stacks such as Datadog, New Relic, Honeycomb, and many others. The project is maintained by Traceloop under the Apache 2.0 license. Getting started is simple: install the Python SDK with 'pip install traceloop-sdk' and initialize with 'Traceloop.init()'.

Key Features

Built on OpenTelemetry, enabling connection to existing observability solutions
Instruments LLM providers: OpenAI, Anthropic, Bedrock, Cohere, Gemini, Mistral, Ollama, and more
Instruments vector databases: Chroma, Pinecone, Qdrant, Weaviate, Milvus, and more
Instruments AI frameworks: LangChain, LlamaIndex, Haystack, CrewAI, LangGraph, and more
Easy setup with 'pip install traceloop-sdk' and one line of code to initialize
Supports 20+ destination platforms including Datadog, New Relic, Google Cloud, Grafana, and others
Open source under Apache 2.0 license

Pros & Cons

Pros
  • Leverages the OpenTelemetry standard, avoiding vendor lock-in
  • Broad coverage of LLM providers, vector databases, and frameworks
  • Easy to get started with minimal code changes
  • Supports many popular observability backends
  • Actively maintained and open source
Cons
  • Python-only SDK (no native JS/TS version, though a separate JS version exists)
  • Requires manual setup for non-default destinations
  • Limited to observability; does not provide evaluation or prompt management features

Best For

Monitor and debug LLM application performance and behaviorTrace calls across multiple LLM providers and vector databases in a single viewIntegrate LLM observability into existing monitoring stacks (e.g., Datadog, Honeycomb)Identify latency, cost, and error patterns in AI workflows

Alternatives to Openllmetry

Arize Phoenix

Open-source AI observability for LLM apps

Langfuse

An open-source LLM engineering platform for tracing, evaluation, prompt management, and metrics. [#opensource](https://github.com/langfuse/langfuse)

FAQ

What is OpenLLMetry?
OpenLLMetry is an open-source set of extensions on top of OpenTelemetry that provides complete observability for LLM applications, including tracing of LLM calls, vector DB queries, and framework operations.
How do I install OpenLLMetry?
Install the Python SDK using pip: 'pip install traceloop-sdk'. Then add 'from traceloop.sdk import Traceloop; Traceloop.init()' to your code.
What destinations are supported?
Supported destinations include Traceloop, Axiom, Azure Application Insights, Braintrust, Datadog, Dynatrace, Google Cloud, Grafana, Honeycomb, New Relic, OpenTelemetry Collector, and many more.
Can I use OpenLLMetry with my existing observability stack?
Yes, because it is based on OpenTelemetry, it can connect to any OTLP-compatible backend or via exporters to specific platforms like Datadog and New Relic.