How It Works
Weights & Biases (W&B) provides a Python-based client library designed to streamline the development lifecycle of machine learning projects. The library integrates with popular frameworks and libraries, enabling users to track, visualize, and manage experiments from datasets to production models. At its core, W&B facilitates experiment tracking by logging hyperparameters, metrics, and artifacts, ensuring reproducibility and collaboration across teams.
To begin using W&B, users install the wandb library via pip and authenticate by creating an API key through the W&B platform. The API key is securely stored, either in a password manager or environment variables, to ensure safe access. Once authenticated, users initialize a run within their Python script or notebook using wandb.init(), specifying a project name and hyperparameters. The with syntax marks the run as complete upon exiting the block, while manual calls to run.finish() are supported in notebook environments.
During training, users log metrics such as accuracy and loss using run.log(), which are then visualized on the W&B platform. The platform organizes runs in a centralized dashboard, allowing users to compare experiments, analyze trends, and debug issues efficiently. W&B also supports artifact versioning, enabling users to track datasets, models, and other files associated with experiments.
W&B operates on a hybrid model, offering both a fully managed cloud platform and self-hosted options. The cloud platform is deployed on Google Cloud Platform (GCP) in North America, providing a multi-tenant or dedicated single-tenant environment. For organizations requiring on-premises or private cloud deployments, W&B Server can be installed on AWS, GCP, Azure, or local infrastructure. This flexibility ensures compliance with organizational security and data residency requirements.
The library supports a range of Python versions, with a commitment to maintaining compatibility for at least six months beyond the official end-of-life date. Additionally, W&B encourages community contributions, providing guidelines for bug reports, feature requests, and code contributions through GitHub Issues and the Contribution Guide.
Use Cases
W&B is widely adopted for experiment tracking and reproducibility in machine learning workflows. Researchers and engineers use it to log hyperparameters, metrics, and artifacts, creating a comprehensive audit trail for experiments. For example, a user training a model can log accuracy and loss metrics, visualize them over epochs, and compare different configurations side by side.
Beyond traditional machine learning, W&B extends its capabilities to large language models (LLMs) through Weave, a suite of tools designed for LLM tracing and evaluation. Weave integrates seamlessly with existing W&B workflows, allowing users to monitor LLM applications, debug outputs, and evaluate performance metrics such as perplexity, token usage, and response quality. This integration reduces the overhead of managing separate tools for LLM development, consolidating observability within a single platform.
W&B also supports hyperparameter sweeps, enabling automated exploration of parameter spaces to optimize model performance. Additionally, W&B’s artifact versioning ensures that datasets, models, and other dependencies are tracked and retrievable, supporting reproducibility across experiments.
For organizations with stringent security or compliance requirements, W&B’s self-hosted deployment options provide a controlled environment for managing sensitive data. This includes adherence to standards such as ISO/IEC 27001, ISO/IEC 27017, ISO/IEC 27018, SOC 2, HIPAA, and GDPR. By deploying W&B Server on private infrastructure, users can maintain full control over data storage and access, aligning with regulatory obligations and internal policies.
Experiment Tracking and Visualization
W&B’s primary function is experiment tracking, where users log hyperparameters, metrics, and system metrics during training. The platform visualizes these metrics in real-time, providing interactive dashboards for analysis. For instance, a metric like train/ can be plotted with a step metric like train/step, offering granular control over visualizations.
The W&B LEET terminal user interface (TUI) enhances local experiment analysis by allowing users to filter console logs, resize panes, and draw guides for line charts.
Artifact and Model Versioning
Artifact versioning is a core capability of W&B, enabling users to track datasets, models, and other files associated with experiments. Users log artifacts using run.log_artifact(), specifying types such as model or dataset. For example, a trained model file (my_model.pt) can be logged as an artifact, making it accessible for future experiments or deployment.
The registry API supports version queries with optional ordering, allowing users to retrieve specific artifact versions based on criteria such as creation date or metadata. This ensures traceability and reproducibility, as users can revert to previous versions of artifacts if needed. W&B also supports collections and registries, enabling hierarchical organization of artifacts across projects.
Hyperparameter Sweeps and Automation
W&B automates hyperparameter sweeps by defining configurations in sweep YAML files, which specify parameter ranges and search strategies. The platform executes sweeps in parallel, logging results for comparison.
Automations can operate at the team or organization level, supporting collaborative pipelines.
Integration with Machine Learning Frameworks
W&B integrates with popular ML frameworks, including PyTorch, TensorFlow, scikit-learn, and XGBoost, simplifying the adoption of experiment tracking. For instance, users can log training metrics directly from PyTorch using wandb.log(), or visualize scikit-learn model performance with wandb.sklearn.plot_classifier().
These integrations reduce the overhead of implementing custom logging solutions, allowing users to focus on model development. W&B’s compatibility with frameworks ensures that existing workflows can be augmented with experiment tracking without significant modifications.
Security and Compliance
W&B prioritizes security and compliance, offering features such as API key management, data encryption, and role-based access control. The platform is certified under ISO/IEC 27001, ISO/IEC 27017, and ISO/IEC 27018, and is compliant with SOC 2 and HIPAA. For organizations handling sensitive data, W&B’s self-hosted deployment options provide an additional layer of control, ensuring data residency and regulatory alignment.
Additionally, W&B supports GDPR requirements for processing personal information, allowing users to manage data subject requests and ensure transparency.
Performance Optimizations
Recent updates to the W&B library have introduced performance improvements, such as gzip compression for filestream requests, which reduces network traffic when logging metrics. This feature is opt-in and requires server support, with plans to make compression the default in future releases.