How it Works
Langtrace is an open-source observability tool designed specifically for Large Language Model (LLM) applications. It captures traces and metrics across model calls, framework interactions, and vector database operations, providing end-to-end visibility into LLM-driven workflows. Built on OpenTelemetry (OTEL) standards, Langtrace ensures compatibility with existing observability ecosystems while introducing specialized semantic conventions tailored for LLM applications. These conventions are actively developed and documented within the project repository, allowing users to contribute feedback and refine the standards over time.
The system operates by instrumenting LLM applications through SDKs for TypeScript and Python. Langtrace supports both managed cloud deployments and self-hosted setups, with the latter ensuring no telemetry data leaves the user’s infrastructure.
For cloud deployments, Langtrace provides a managed SaaS version accessible via a web interface. Users sign up, create projects to organize traces, and generate API keys for SDK initialization. The SDKs must be initialized before any LLM module imports to ensure accurate trace capture. Self-hosted deployments require running three core services: a Next.js frontend, a PostgreSQL database for metadata storage, and a ClickHouse database for traces, metrics, logs, and spans. Docker and Docker Compose are used to streamline deployment, with configuration options available via an environment file (.env).
PostgreSQL handles metadata and user management, while ClickHouse stores time-series data efficiently. Authentication methods include admin passwords, Google OAuth, and Azure AD OAuth, with detailed configuration guides provided for self-hosted setups. Video tutorials and step-by-step documentation further simplify deployment across environments like Docker or Railway.
Traces generated by Langtrace include semantic span attributes that provide context-specific insights. For example, recent updates have added support for displaying Cleanlab trustworthiness scores within traces, alongside additional metadata such as explanation logs.
Use Cases
Langtrace addresses critical observability challenges in LLM applications, particularly in environments where performance, cost, and debugging are priorities. This is especially valuable in production systems where LLM-driven features directly impact user experience.
The tool’s integration with frameworks like LangChain and LlamaIndex further extends its utility to applications built on these libraries.
Self-hosted deployments of Langtrace are ideal for organizations with strict data residency requirements or those developing proprietary LLM applications. By running Langtrace on-premises or in private clouds, users ensure that all traces, metrics, and logs remain within their control.
The tool’s extensible architecture allows for custom instrumentation, enabling researchers to capture domain-specific metrics. Additionally, the project’s open-source nature fosters collaboration, with contributions from developers improving integrations, fixing bugs, and adding features like Cleanlab support.
Key Features & Capabilities
The project defines semantic conventions for LLM-specific traces, including attributes for model providers, frameworks, and vector databases. These conventions standardize how traces are structured, making it easier to analyze and correlate data across different components of an LLM application.
The system captures metrics such as request latency, error rates, and throughput, presenting them in a self-hostable UI.
By analyzing traces and metrics, developers can identify performance bottlenecks, such as slow vector database queries or inefficient model prompting strategies.
The ability to filter and search traces further streamlines debugging sessions.
The self-hostable UI ensures that these analytics remain accessible without relying on external services.
Integration support is a major differentiator for Langtrace, with SDKs available for TypeScript and Python. The TypeScript SDK supports LLM providers such as OpenAI, Anthropic, Azure OpenAI, Cohere, DeepSeek, xAI, Groq, Perplexity, Gemini, and AWS Bedrock, as well as frameworks like LlamaIndex, VertexAI, and Vercel AI. The Python SDK extends this coverage to additional providers like Mistral and frameworks such as LangChain, LiteLLM, and GuardrailsAI.
Self-hosting is a key capability, offering organizations full control over their observability infrastructure. The deployment process involves running three services via Docker Compose: a Next.js application for the UI, PostgreSQL for metadata storage, and ClickHouse for traces and logs. Authentication methods include admin passwords, Google OAuth, and Azure AD OAuth, with configuration guides provided for each. Users can deploy Langtrace on platforms like Railway with minimal setup, and the project’s documentation includes video tutorials to assist with complex configurations. No telemetry data is collected in self-hosted mode, ensuring privacy and compliance.
Recent releases have focused on refining the user experience, such as correcting setup instructions for the TypeScript SDK and addressing self-hosting issues related to team configurations. Contributions from the community drive these enhancements, with developers encouraged to join the project’s Discord workspace or GitHub discussions to propose features or report issues. The license model distinguishes between the AGPL 3.0 license for the core application and the Apache 2.0 license for the SDKs, ensuring flexibility for different use cases.