OpenTelemetry-Based Observability for AI Systems
Pydantic Logfire is an observability platform designed to provide comprehensive visibility into AI-driven applications, particularly large language models (LLMs) and agent systems. Developed by the team behind Pydantic, it integrates seamlessly with OpenTelemetry (OTel), an open standard for collecting and exporting telemetry data. This ensures compatibility with existing observability tooling while offering specialized features for AI workloads.
The platform captures structured traces, metrics, and logs, feeding them into the Logfire backend. Unlike traditional observability tools that focus solely on backend infrastructure, Logfire correlates AI-specific events—such as model invocations, tool calls, and retrieval steps—with backend operations like database queries and API requests. This end-to-end tracing helps identify bottlenecks, cost inefficiencies, and failures across the entire stack.
Its dashboard is intuitive yet capable of handling complex queries, making it accessible to engineers across teams. The platform supports Python-centric insights, including detailed visualization of Python objects, event-loop telemetry, and profiling of code execution. This level of granularity is particularly valuable for debugging AI agents, where reasoning steps and tool interactions are critical.
The SDK is open-source under the MIT License, while the Logfire platform (UI and backend) remains closed-source. Users can self-host the platform with an enterprise license, ensuring compliance with data residency requirements and reducing vendor lock-in. The open-source SDKs for Python, TypeScript, and Rust allow data export to any OTel-compatible backend, further enhancing flexibility.
Logfire’s integration with OpenTelemetry extends its reach beyond Python. It supports JavaScript/TypeScript, Rust, and any language with OTel instrumentation, including Go, Java, .NET, and Ruby. This broad compatibility ensures that AI applications built on frameworks like FastAPI, LangChain, or LlamaIndex can be monitored without re-instrumentation. Prebuilt integrations for over 100 infrastructure components—ranging from cloud services to on-prem databases—simplify setup for production environments.
The platform’s architecture is built on FusionFire, a columnar data engine derived from Apache DataFusion and optimized for wide, nested telemetry data. This engine enables efficient querying of traces using Postgres-flavored SQL, allowing users to join spans, aggregate metrics, and analyze attributes without proprietary query languages. SQL support also integrates with existing BI tools and databases, reducing the learning curve for teams familiar with relational queries.
While those platforms emphasize prompt management and model behavior, Logfire provides full-stack observability, correlating AI events with backend performance. For example, a slow vector database query in a RAG pipeline can be traced back to its impact on LLM latency, enabling targeted optimizations. This holistic approach is critical for diagnosing issues that span multiple layers of an AI system.
One of Logfire’s standout features is its support for AI evaluations. The platform allows teams to define automated judges for deterministic checks—such as JSON validity, citation presence, or response length—while delegating subjective assessments (e.g., tone or relevance) to human reviewers. This methodology aligns with best practices from organizations like Airbnb, which advocate for iterative evaluation based on real-world failures.
Security and compliance are central to Logfire’s design. The platform adheres to SOC 2 Type II, HIPAA, and GDPR standards, with additional controls for EU data residency and PII scrubbing. Role-based access, SSO integration, and audit logs provide granular governance, while Business Associate Agreements (BAAs) are available for enterprise customers. For organizations handling sensitive data, Logfire offers self-hosted deployment options, ensuring full control over data storage and processing.
Users receive 10 million spans, logs, and metrics free per month with no credit card requirement. Additional records are billed at $2 per million, with optional price caps for cost predictability. As of mid-2026, the platform processes over 200 billion records monthly, demonstrating its ability to handle enterprise-scale telemetry.
How It Works
Logfire’s functionality is divided into two primary components: the open-source SDK and the closed-source platform. The SDK, available for Python, TypeScript, and Rust, instruments applications by wrapping OpenTelemetry’s signals (traces, metrics, and logs). Users authenticate via the logfire auth CLI command, then configure the SDK with logfire.configure(). Instrumentation can be manual—using spans and logs for custom events—or automatic, via integrations with frameworks like FastAPI, LangChain, or Pydantic AI.
This automatically captures HTTP requests, response times, and Pydantic model validations. Similarly, AI agents built with Pydantic AI or LangChain emit traces that include model inputs, outputs, tool calls, and reasoning steps. These traces are then correlated with backend services—such as databases or external APIs—creating a unified view of the request lifecycle.
The Logfire platform processes and stores telemetry data using FusionFire, its columnar engine. Queries are executed in Postgres-flavored SQL, enabling complex analyses such as identifying slow spans, aggregating LLM costs by model, or comparing evaluation scores across agent runs. The UI presents these results in purpose-built views for LLMs and agents, displaying inventories, token usage, and cost metrics alongside traditional observability data. This dual perspective accelerates debugging by linking AI-specific events to infrastructure performance.
Logfire also supports evaluations as first-class citizens. For instance, a team might run offline evaluations on a dataset to catch edge cases missed in production, then use the same queries to monitor live performance.
Use Cases
Logfire is tailored for organizations deploying AI agents, RAG systems, or LLM-powered applications at scale. Its full-stack observability addresses pain points that arise when AI components interact with backend services, such as latency in vector databases, API timeouts, or memory leaks in background workers.
In RAG pipelines, Logfire traces the entire workflow: from user queries to vector database retrieval, context chunking, and LLM generation. Slow retrievals or irrelevant chunks are visible alongside the final output, enabling teams to optimize retrieval strategies or refine prompt engineering. Similarly, Logfire’s cost tracking—measured in tokens and API calls—identifies expensive model invocations or inefficient tool usage, directly impacting operational budgets.
AI agents benefit from Logfire’s purpose-built views, which display tool inventories, reasoning steps, and decision logic alongside backend traces. This visibility is critical for debugging complex workflows, such as those involving multi-step tool execution or conditional branching. Logfire’s evaluations further enhance agent reliability by catching failures like malformed JSON responses or tool calls with incorrect arguments.
Logfire’s compatibility with OpenTelemetry extends its utility to mixed-language environments. For instance, a Python backend serving a TypeScript frontend—both instrumented with Logfire—can be traced as a single request. This is particularly valuable for AI gateways or edge workers, where latency across services directly impacts user experience. By preserving OTel portability, Logfire also allows data forwarding to SIEMs or data warehouses, avoiding vendor lock-in while maintaining centralized observability.
Security-sensitive applications, such as those handling PII or regulated data, leverage Logfire’s compliance features. The platform supports edge redaction for sensitive fields, role-based access controls, and audit logging. For organizations subject to HIPAA or GDPR, Logfire provides BAAs and EU data residency options, along with independently audited SOC 2 controls. Self-hosting further ensures that telemetry data remains within organizational boundaries, addressing concerns around third-party access.
Key Features and Capabilities
- OpenTelemetry Integration: Logfire builds on OTel’s signals (traces, metrics, logs) and integrations, ensuring compatibility with existing observability stacks. This includes support for over 100 infrastructure components and frameworks like FastAPI, Django, and LangChain. OTel’s portability allows data export to multiple backends, such as Datadog or Grafana, without re-instrumentation.
- SQL Querying: The platform’s backend uses FusionFire, a columnar engine optimized for wide, nested telemetry data. Users query traces with Postgres-flavored SQL, enabling joins, aggregations, and JSON operators. This eliminates the need for proprietary query languages and integrates with existing BI tools. Example queries include identifying slow spans, calculating LLM costs by model, or analyzing p95 latency by route.
- AI-Specific Observability: Logfire correlates AI events (model calls, tool invocations, evaluations) with backend operations (database queries, API hops). Purpose-built views for LLMs and agents display token usage, cost metrics, and reasoning steps alongside traditional traces. This holistic approach reveals bottlenecks that span multiple layers, such as a slow vector search delaying context retrieval in a RAG pipeline.
- Evaluations and Judges: Teams define automated judges for deterministic checks (e.g., JSON validity) and human judges for subjective assessments (e.g., tone or relevance). Evaluation results are stored as span attributes, enabling SQL-based comparisons between model versions or datasets. This methodology supports iterative improvement, where production failures inform offline evaluations and vice versa.
- Security and Compliance: Logfire adheres to SOC 2 Type II, HIPAA, and GDPR standards, with additional controls for EU data residency and PII scrubbing. Role-based access, SSO integration, and audit logs provide governance, while BAAs and self-hosting options cater to enterprise requirements. The platform’s subprocessor list and independently audited controls are available upon request.
- Additional records are billed at $2 per million, with optional price caps for predictable costs. As of mid-2026, the platform processes over 200 billion records monthly, demonstrating its ability to handle high-volume AI workloads without performance degradation.
- Self-Hosting and Open-Source SDKs: While the Logfire platform is closed-source, its SDKs for Python, TypeScript, and Rust are open-source under the MIT License. Users can self-host the platform with an enterprise license, ensuring data remains on-premises. The open-source SDKs also allow data export to any OTel-compatible backend, further reducing dependency on the Logfire service.
Implementation Examples
For Python applications, installation begins with pip install logfire, followed by authentication via logfire auth. Manual tracing involves configuring the SDK with logfire.configure() and logging events using methods like logfire.info() or logfire.span(). For example, a basic trace might log a user input and its processing steps, with nested spans for tool calls or database queries.
Automatic instrumentation is equally straightforward. Similarly, AI agents built with Pydantic AI or LangChain emit traces automatically, correlating model calls with backend services. This reduces boilerplate while ensuring comprehensive coverage of the application stack.
For teams using existing observability tools, Logfire’s OTel compatibility allows parallel ingestion. Data can be forwarded to SIEMs, warehouses, or dashboards without disrupting current workflows. This flexibility is particularly valuable during migrations or when combining Logfire’s AI-specific features with traditional monitoring.
For instance, identifying the slowest spans in an agent run involves a simple SQL query, while calculating LLM costs by model requires aggregating token usage attributes. These queries can be executed in the Logfire UI, via MCP (Machine-to-Machine) server, or directly in supported BI tools.
Addressing Common Challenges
Logfire’s design addresses several challenges in AI observability. First, it bridges the gap between LLM-focused tools and full-stack monitoring by correlating AI events with backend performance. This is critical for diagnosing issues like slow retrievals in RAG pipelines or tool failures in agent workflows, which often require cross-layer analysis. Second, its SQL-based querying eliminates the need for proprietary languages, reducing the learning curve for teams already using relational databases.
Security and compliance are integrated from the ground up. Logfire’s support for PII scrubbing, role-based access, and audit logs aligns with enterprise requirements, while self-hosting options provide additional control. The platform’s adherence to SOC 2 and HIPAA further simplifies compliance for regulated industries.