How MLRun Works
MLRun is an open-source AI orchestration platform designed to streamline the development and management of continuous AI applications across their lifecycle. It integrates into existing development and CI/CD environments, automating the delivery of production data, machine learning pipelines, and online applications. By reducing engineering efforts, time to production, and computational resource requirements, MLRun enables teams to collaborate effectively across data, machine learning, software, and DevOps/MLOps domains. The platform supports a wide range of IDEs, both locally and in the cloud, and breaks down traditional silos to facilitate continuous improvements.
MLRun operates on an open architecture that supports various data stores, development tools, services, and platforms. Its core components include project management, functions, data and artifacts management, batch runs and workflows, real-time serving pipelines, model monitoring, alerts and notifications, and a feature store. These components work together to provide a unified framework for managing AI workflows from data ingestion to model deployment and monitoring.
The platform automates data preparation, model tuning, validation, and optimization, while also enabling the rapid deployment of scalable real-time serving and application pipelines. MLRun supports multi-cloud, hybrid, and on-prem environments, ensuring flexibility in deployment. It provides built-in observability, allowing teams to monitor models, resources, and data in real time, and to auto-trigger alerts, retraining, and customization processes for generative AI models.
MLRun’s architecture is designed to optimize resource utilization, reducing compute costs while scaling workloads dynamically. It supports both batch and real-time data processing, structured and unstructured data, and integrates with mainstream frameworks, managed ML services, and third-party tools. The platform’s feature store automates the collection, transformation, storage, cataloging, and serving of data features, enabling reuse and sharing across the ML lifecycle.
At its core, MLRun manages projects as centralized repositories for assets, metadata, and services such as data, functions, jobs, artifacts, models, and secrets. Projects can be versioned, mapped to Git repositories, and integrated with IDEs like PyCharm or VSCode, facilitating collaboration and CI/CD workflows. Access controls ensure that projects can be restricted to specific users and roles, enhancing security and governance.
Functions in MLRun are automatically deployed software packages with configurable runtime attributes, such as images, libraries, commands, arguments, and resource allocations. These functions can be executed as batch jobs or real-time pipelines, with results and artifacts tracked for comparison and analysis. The platform’s real-time serving pipeline enables the deployment of scalable data and ML pipelines, handling API requests, data preparation, model inference, and action-driven workflows.
Model monitoring is a built-in feature of MLRun, providing observability into data, models, resources, and production components. It supports the identification of drift, performance anomalies, and data quality issues, while enabling governance, explainability, and automated retraining. Alerts and notifications ensure that teams are promptly informed of potential issues, allowing for proactive intervention and continuous optimization of AI applications.
MLRun’s open architecture ensures compatibility with mainstream frameworks, managed ML services, and third-party tools, making it adaptable to diverse enterprise environments. Its support for multi-cloud, hybrid, and on-prem deployments further enhances its versatility, allowing organizations to deploy workloads according to their specific infrastructure requirements. The platform’s emphasis on automation, observability, and collaboration positions it as a comprehensive solution for managing AI workflows end-to-end.
Use Cases
MLRun addresses a broad spectrum of AI and MLOps tasks, including data management, model development, deployment, and monitoring. Its capabilities extend to generative AI workflows, enabling teams to develop, scale, deploy, and monitor AI models across enterprise environments. The platform’s support for both structured and unstructured data, batch and real-time processing, and feature reuse makes it suitable for a variety of applications, from call center automation to banking agent systems.
In data management, MLRun supports large-scale batch or real-time data processing, data lineage, and versioning. It integrates with vector databases and provides guardrails for data management, ensuring that inappropriate data is filtered out early to save computational resources. For example, MLRun can be used to process unstructured data using large language models (LLMs), enabling applications such as call center automation where LLMs analyze and respond to customer inquiries in real time.
For model development, MLRun automates the creation of ML pipelines that collect, preprocess, and train data using scalable services. It supports custom or third-party training services and allows teams to bring their own pre-trained models into the pipeline. Experiment tracking, model registration, versioning, and deployment are all managed within MLRun, ensuring reproducibility and traceability. Tutorials and demos, such as those for banking agent systems, illustrate how MLRun can be used to fine-tune LLMs and integrate them into production workflows.
Deployment in MLRun is facilitated through serverless functions that enable the productization of trained models as scalable, real-time applications. The platform’s serving capabilities include auto-scaling, GPU utilization optimization, and real-time inference pipelines that handle data enrichment, model serving, and action-driven workflows. For instance, MLRun can deploy LLMs as serverless functions, allowing them to scale dynamically based on demand while maintaining low latency and high performance.
Live operations monitoring in MLRun provides real-time observability into resources, data, model performance, and business KPIs. It collects production data, metadata, and metrics to identify risks, control costs, and measure business impact. Model monitoring features detect drift, performance degradation, and data quality issues, triggering alerts or retraining jobs as needed.
MLRun’s project management and CI/CD automation features enable teams to organize assets, metadata, and services into version-controlled projects. These projects can be mapped to Git repositories or IDEs, facilitating collaboration and automated workflows. Access controls allow for role-based restrictions, ensuring that projects are secure and compliant with organizational policies. For example, a team can use MLRun to automate the CI/CD pipeline for an ML model, from data ingestion to deployment, while maintaining full traceability and reproducibility.
The platform’s feature store automates the collection, transformation, storage, cataloging, and serving of production data features. This enables feature reuse across offline development and real-time deployment, reducing redundancy and improving efficiency. For instance, a feature store can be used to serve preprocessed data to multiple models, ensuring consistency and reducing the need for redundant data preparation steps.
MLRun’s integration with tools like NVIDIA NIM and its support for multi-cloud and hybrid environments make it adaptable to enterprise-scale AI deployments. Demos and tutorials, such as those for deploying generative AI models in production, showcase how MLRun can be used to transform enterprise operations. By providing a unified technology stack for data engineers, data scientists, and machine learning engineers, MLRun breaks down silos and drives collaboration, reducing maintenance times and increasing the robustness of AI applications.
Key Features and Capabilities
MLRun’s project management system serves as a centralized control layer for organizing and managing assets such as data, functions, jobs, workflows, secrets, and artifacts. Projects can be imported or exported as a whole, mapped to Git repositories, and integrated with IDEs like PyCharm or VSCode. This integration enables versioning, collaboration, and seamless CI/CD workflows, while access controls ensure that projects are restricted to authorized users and roles.
Functions in MLRun are deployable software packages that encapsulate one or more methods and runtime-specific attributes. These attributes include images, libraries, commands, arguments, and resource allocations, allowing functions to be executed as batch jobs or real-time pipelines. MLRun’s functions can be used to process data, train models, or serve predictions, with results and artifacts tracked for comparison and analysis. This modular approach enables teams to build complex workflows by combining multiple functions into cohesive pipelines.
Data and artifacts management in MLRun provides glueless connectivity to various data sources, supporting both structured and unstructured data. The platform automates data ingestion, processing, lineage tracking, and versioning, ensuring that data is accurately cataloged and reusable across workflows. The feature store component further enhances this capability by automating the collection, transformation, storage, and serving of production data features, enabling reuse in both offline and real-time environments.
Batch runs and workflows in MLRun allow teams to execute one or more functions with specific parameters, collecting and tracking results and artifacts for comparison. This capability supports the automation of ML pipelines, from data preprocessing to model evaluation and deployment. For example, a batch workflow might include steps for data preprocessing, model training, and evaluation, with each step’s results logged and versioned for reproducibility.
Real-time serving pipelines in MLRun enable the rapid deployment of scalable data and ML pipelines using serverless technology. These pipelines handle API requests, data preparation, model inference, and action-driven workflows, with built-in auto-scaling to manage varying loads. The platform supports GPU utilization optimization, ensuring that models are served efficiently even under high demand. For instance, a real-time serving pipeline might process incoming requests, enrich data with contextual information, and serve predictions using one or more models, all while maintaining low latency and high availability.
Model monitoring in MLRun provides observability into data, models, resources, and production components. It tracks model performance, detects drift, and identifies data quality issues, triggering alerts or retraining jobs as needed. Custom application metrics can be defined to measure business impact, and governance features ensure that models remain explainable and compliant with organizational policies. For example, MLRun can monitor a deployed LLM for performance degradation, automatically triggering a retraining job if drift is detected.
Alerts and notifications in MLRun ensure that teams are promptly informed of potential issues or status changes in runs and pipelines. These alerts can be configured to identify problems such as data drift, model performance degradation, or resource constraints, enabling proactive intervention. Notifications provide updates on the status of runs and pipelines, ensuring transparency and accountability. For instance, an alert might notify a team when a model’s accuracy drops below a specified threshold, prompting an investigation into the cause.
The feature store in MLRun automates the management of production data features, enabling their reuse across offline development and real-time deployment. It collects, prepares, catalogs, and serves features with minimal engineering effort, reducing redundancy and improving efficiency. Features can be shared across teams and workflows, ensuring consistency and reducing the need for redundant data preparation. For example, a feature store might serve preprocessed customer data to multiple models, enabling personalized recommendations in real time.
MLRun’s support for mainstream frameworks, managed ML services, and third-party tools ensures compatibility with a wide range of AI and data processing technologies. Its open architecture allows teams to integrate MLRun with existing tools and services, while its support for multi-cloud, hybrid, and on-prem environments ensures flexibility in deployment. This adaptability makes MLRun suitable for enterprise-scale AI applications, where integration with legacy systems and compliance with organizational policies are critical.
Recent updates to MLRun, such as those in versions 1.12.0 and 1.13.0-rc9, have focused on security enhancements, dependency updates, and bug fixes. For example, version 1.12.0 addressed vulnerabilities in Jupyter images, updated dependencies to remediate CVEs, and enforced non-root pod security contexts.