An Orchestration Runtime for Production-Grade Multi-Agent AI Systems
Orloj is an open-source orchestration runtime designed to transform multi-agent AI systems from experimental prototypes into production-grade operations. Unlike traditional agent frameworks that focus solely on building or collaborating agents, Orloj provides the infrastructure layer required to govern, schedule, and observe agent systems at scale. By defining agents, tools, policies, and workflows in declarative YAML manifests, Orloj eliminates the need for bespoke orchestration code, ensuring deterministic execution and fail-closed governance.
The runtime spans multiple layers essential for developing, running, and governing agentic systems. These include interfaces for YAML manifests, REST APIs, SDKs, and Kubernetes CRDs; agent definitions with prompts, graph topology, roles, and execution contracts; execution runtimes with sequential and message-driven workflows; model and context layers for routing, fallback models, and memory management; tool and integration layers for HTTP, gRPC, webhooks, and WASM interoperability; governance and human review mechanisms like policies, roles, and approval gates; and observability tools such as traces, logs, metrics, and UI views. This full-stack approach ensures that every component of an agent system is treated as a versioned resource with operational visibility and lifecycle management.
Orloj is actively under development, with APIs and resource schemas subject to change before reaching version 1.0. The project is licensed under Apache 2.0, making it permissive for both open-source and commercial use.
How It Works
Orloj operates as a full-stack orchestration platform, where agents, tools, and policies are declared as YAML resources and managed through a unified API. The system follows a declarative model, where users define agents, systems, tools, model endpoints, memory, tasks, secrets, evaluations, and policies in YAML manifests. This approach ensures consistency and operational control over agent systems.
The runtime architecture is divided into distinct phases: declaration, reconciliation, scheduling, execution, and governance. During the declare phase, users write YAML resources for all components of their agent system. Finally, the govern phase enforces policies, roles, tool permissions, and approval gates, ensuring fail-closed operations where unauthorized actions are denied by default.
Session checkpointing allows agent loops to persist versioned snapshots at safe ReAct step boundaries, enabling recovery from lease loss and providing APIs to inspect, verify, replay, rewind, or fork sessions. Durable interactive sessions bind multi-turn conversations to agent systems, serializing user turns, persisting replayable events, and supporting interruption, pause/resume, cancellation, and idle expiry. These features enhance reliability and auditability, making Orloj suitable for production environments where operational resilience is critical.
The platform supports both local development and distributed deployments. Users can start with an embedded worker in a single process and scale to PostgreSQL, NATS JetStream, and distributed workers as needed. This ensures full visibility into agent operations, enabling debugging, performance tuning, and compliance auditing.
Use Cases
Orloj is designed for teams transitioning from agent demonstrations to production systems requiring governance, scheduling, and observability. It addresses key pain points in multi-agent AI operations, including resource drift, hidden handoffs, silent failures, ungoverned tool access, and scattered debugging logs. By centralizing agent, tool, model, and workflow configurations in version-controlled manifests, Orloj eliminates configuration drift and ensures consistency across environments.
This structured approach ensures that workflows are deterministic and auditable, reducing the risk of errors or data loss. For example, a research team might use Orloj to orchestrate a hierarchical system where a researcher queries CVE databases, an analyst processes findings, and an editor approves outputs—each with enforced token budgets and tool permissions.
Another critical use case is tool governance, where agents must interact with external services under strict access controls. Orloj enforces tool permissions, isolation, authorization, approval gates, retry policies, and audit-friendly traces. This ensures that agents only access tools they are permitted to use, with all actions logged for compliance and security purposes. For instance, agents scanning infrastructure for exposed secrets can operate with metadata-only access, auditing secrets without reading their values.
Orloj also excels in scalable and reliable agent operations, where failure handling, retries, and dead-letter management are critical. The runtime includes lease-based task ownership, retry mechanisms with jitter, idempotency tracking, and dead-letter queues to prevent cascading failures. This makes agent fleets resilient to partial outages, ensuring continuity even when individual components fail.
For compliance and auditability, Orloj provides structured logging of all agent actions, tool calls, and policy decisions. This structured audit trail supports compliance workflows for frameworks like HIPAA, SOC 2, and the EU AI Act. Governance is enforced at the execution layer, ensuring that unauthorized actions are denied by default (fail-closed) rather than allowed unless explicitly blocked (fail-open). This aligns with production-grade operational requirements where security and compliance are non-negotiable.
Key Features & Capabilities
Orloj’s architecture is built around several core features that address the operational challenges of multi-agent AI systems. These include declarative YAML manifests, which define agents, tools, policies, and workflows as version-controlled resources. For example, a single orlojctl apply -f ./your-system/ command reconciles all components of an agent system, ensuring consistency across deployments.
The runtime enforces fail-closed governance, where policies and permissions are evaluated inline on every agent turn and tool call. Unauthorized actions are denied by default, with structured errors and complete audit traces generated for every decision. This ensures that agents operate within predefined boundaries, reducing the risk of unintended actions or security breaches. For instance, an agent attempting to access a restricted tool will receive a denial with a traceable explanation, rather than silently proceeding.
Orloj supports deterministic execution through structured workflows defined in YAML graphs. These graphs enable fan-out and fan-in patterns, allowing agents to collaborate in pipelines or hierarchies with explicit handoffs. For example, a pipeline agent might check contracts against regulatory requirements, while a model whitelist ensures sensitive content is only processed by approved providers. Every finding is traced and auditable, supporting compliance and debugging efforts.
The platform includes reliability primitives such as lease-based task ownership, retry mechanisms, idempotency tracking, and dead-letter handling. These features prevent cascading failures and ensure that agent fleets remain operational even during partial outages. For example, if a worker loses its lease, Orloj can recover from the latest compatible checkpoint, ensuring no work is lost or duplicated. Similarly, retries with jitter prevent thundering herds, while dead-letter queues capture failed tasks for later inspection.
Orloj provides comprehensive observability through structured logging, distributed tracing, metrics collection, and cost attribution. Users can trace an agent’s decision path, monitor tool calls, analyze latency, and allocate costs by agent or workflow. This visibility is critical for debugging, performance tuning, and compliance auditing. For example, the Console trace waterfall renders real spans grouped into collapsible lanes, showing latency and duration for each operation.
The runtime also supports interactive sessions and agent-to-agent (A2A) communication. Interactive sessions bind multi-turn conversations to agent systems, persisting replayable events and supporting features like interruption, pause/resume, and cancellation. A2A communication allows agents to discover and invoke remote agents via JSON-RPC, with support for public and bearer-authenticated systems. This enables distributed agent ecosystems where specialized agents can collaborate across organizational boundaries while maintaining governance and security.
Tools run in isolated containers with explicit image pins, ensuring reproducibility and security. For example, WASM-isolated tools can scan infrastructure for secrets without accessing sensitive data, while gRPC tools benefit from SSRF protection to prevent security vulnerabilities. The runtime also supports model provider routing, fallback models, and token budgets, ensuring cost-effective and reliable model interactions.
Finally, Orloj is designed for scalability and flexibility, supporting deployments from single-process local development to distributed Kubernetes environments. This ensures that users can start small and scale as needed, with minimal operational overhead. For example, a team might begin with an embedded worker for local testing and later migrate to a distributed setup with persistent storage and messaging for production use.