Catalog
technology#Machine Learning#Platform#Data#DevOps

MLflow

Open-source platform for managing the machine learning lifecycle, offering experiment tracking, model registry and deployment integrations.

MLflow is an open-source platform to manage the end-to-end machine learning lifecycle, including experiment tracking, model packaging, and deployment.
Established
Medium

Classification

  • Medium
  • Technical
  • Technical
  • Intermediate

Technical context

TensorFlow, PyTorch, scikit-learnCloud storage (S3, GCS) and data platformsCI/CD tools (Jenkins, GitHub Actions) and orchestration (Kubernetes)

Principles & goals

Clear separation of tracking, registry and servingEnsure reproducibility and traceabilityFavor easy integration into existing workflows
Build
Team, Domain

Use cases & scenarios

Compromises

  • Lack of governance can lead to disordered model registries
  • Insecure artifact storage can jeopardize privacy and compliance
  • Incompatible framework versions can prevent reproducibility
  • Use consistent naming conventions for experiments and models
  • Automate registration and promotion in CI/CD pipelines
  • Use persistent, versioned artifact storage with access controls

I/O & resources

  • Training code and scripts
  • Training data and artifacts
  • Access to artifact storage and tracking server
  • Traceable run logs and metrics
  • Versioned model artifacts in the registry
  • Deployed model endpoints

Description

MLflow is an open-source platform to manage the end-to-end machine learning lifecycle, including experiment tracking, model packaging, and deployment. It provides lightweight APIs and a tracking server to record runs, artifacts, and model versions. MLflow integrates with common ML frameworks and supports reproducible workflows across teams.

✔

  • Standardized capture of experiment data and artifacts
  • Simplified model versioning and promotion
  • Wide integration with ML frameworks and deployment options

✖

  • Requires infrastructure for tracking server and artifact storage
  • Not all deployment adapters are equally well maintained for every platform
  • By itself not a complete solution for model monitoring

  • Number of logged runs

    Number of training runs recorded in MLflow over a period.

  • Model registrations

    Count of model versions recorded in the registry.

  • Time-to-deploy

    Time from model registration to production deployment.

Open-source project integration

A research team uses MLflow for experiment reproducibility and shares model artifacts via the registry.

Enterprise model platform

A platform automation connects MLflow with Kubernetes for scalable serving and rollbacks.

CI-driven training

CI jobs run trainings, log runs to MLflow and automatically promote validated models.

1

Install and configure the MLflow server and artifact storage.

2

Instrument training scripts with the MLflow tracking API.

3

Integrate the registry into CI/CD pipeline and deployment workflows.

⚠️ Technical debt & bottlenecks

  • Outdated integrations to specific deployment providers
  • Unstructured tagging and naming conventions in the registry
  • Manual deployment steps instead of automated pipelines
artifact-storagedeployment-connectorsversion-management
  • Only local runs without centralized team tracking
  • Registering models but not storing validation or test metadata
  • Using the registry as a backup for raw data
  • Underestimating storage needs for artifacts
  • Overlooked compatibility issues between ML framework versions
  • Missing access control on registry and artifacts
Basic knowledge in machine learning and model trainingExperience with DevOps/platform tools for deploymentKnowledge in configuring artifact storage and permissions
Reproducibility of experimentsScalability of model servingAuditability and traceability
  • • Requires persistent artifact storage (e.g. S3, NFS)
  • • Network access between training environment and tracking server required
  • • Compatibility requirements with used ML frameworks