MLOps Basics Skill

Learn: Master the foundations of Machine Learning Operations for production ML systems.

Skill Overview

Learning Objectives

After completing this skill, you will be able to:

1. Explain the complete ML lifecycle from data to production
2. Assess organizational MLOps maturity levels (0-4)
3. Compare and select appropriate MLOps tools
4. Design basic ML pipelines following best practices
5. Implement foundational MLOps practices in your team

Topics Covered

Module 1: ML Lifecycle Fundamentals (8 hours)

┌─────────────────────────────────────────────────────────────────┐
│                     ML LIFECYCLE PHASES                          │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  ┌──────────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐     │
│  │   Data   │──▶│  Model   │──▶│  Deploy  │──▶│ Monitor  │     │
│  │Collection│   │ Training │   │          │   │          │     │
│  └──────────┘   └──────────┘   └──────────┘   └──────────┘     │
│       │                                              │          │
│       └──────────── Feedback Loop ◀──────────────────┘          │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Topics:

• Data Engineering vs Data Science vs ML Engineering
• Feature Engineering lifecycle
• Model training and evaluation
• Deployment patterns
• Continuous monitoring and retraining

Exercises:

• Map an existing ML project to lifecycle phases
• Identify gaps in current workflow
• Create lifecycle documentation template

Module 2: MLOps Principles (8 hours)

Core Principles (2024-2025 Best Practices):

Exercises:

• Audit current project against principles
• Create reproducibility checklist
• Design testing strategy

Module 3: Tool Ecosystem (12 hours)

Tool Categories & Recommendations:

MLOPS_TOOLS = {
    "experiment_tracking": {
        "open_source": ["MLflow", "DVC"],
        "managed": ["Weights & Biases", "Neptune", "Comet"]
    },
    "feature_stores": {
        "open_source": ["Feast"],
        "managed": ["Tecton", "Hopsworks", "Vertex Feature Store"]
    },
    "orchestration": {
        "open_source": ["Airflow", "Prefect", "Dagster"],
        "managed": ["Kubeflow", "Vertex Pipelines", "SageMaker Pipelines"]
    },
    "model_serving": {
        "open_source": ["TorchServe", "BentoML", "Seldon"],
        "managed": ["Triton", "SageMaker Endpoints", "Vertex Endpoints"]
    },
    "monitoring": {
        "open_source": ["Evidently", "NannyML"],
        "managed": ["WhyLabs", "Arize", "Fiddler"]
    }
}

Exercises:

• Install and configure MLflow locally
• Track an experiment with parameters and metrics
• Register a model in the model registry

Module 4: Best Practices & Patterns (12 hours)

Design Patterns:

1. Training Pipeline Pattern

   Data → Validate → Transform → Train → Evaluate → Register
   

2. Serving Pipeline Pattern

   Request → Preprocess → Predict → Postprocess → Response
   

3. Monitoring Pattern

   Production Data → Compare vs Reference → Detect Drift → Alert/Retrain
   

Exercises:

• Design a training pipeline for a sample project
• Implement basic data validation with Great Expectations
• Set up a simple monitoring dashboard

Code Templates

Template 1: MLflow Experiment Setup

# templates/mlflow_setup.py
import mlflow
from mlflow.tracking import MlflowClient

def setup_experiment(
    experiment_name: str,
    tracking_uri: str = "sqlite:///mlflow.db"
) -> str:
    """Initialize MLflow experiment with best practices."""
    mlflow.set_tracking_uri(tracking_uri)

    # Create or get experiment
    client = MlflowClient()
    experiment = client.get_experiment_by_name(experiment_name)

    if experiment is None:
        experiment_id = client.create_experiment(
            name=experiment_name,
            tags={"version": "1.0", "team": "ml-platform"}
        )
    else:
        experiment_id = experiment.experiment_id

    mlflow.set_experiment(experiment_name)
    return experiment_id


def log_run(params: dict, metrics: dict, model_path: str):
    """Log a complete training run."""
    with mlflow.start_run():
        mlflow.log_params(params)
        mlflow.log_metrics(metrics)
        mlflow.log_artifact(model_path)

Template 2: MLOps Maturity Assessment

# templates/maturity_assessment.py
from dataclasses import dataclass
from enum import IntEnum
from typing import List

class MaturityLevel(IntEnum):
    AD_HOC = 0
    REPEATABLE = 1
    RELIABLE = 2
    SCALABLE = 3
    OPTIMIZED = 4

@dataclass
class AssessmentQuestion:
    dimension: str
    question: str
    level_0: str
    level_1: str
    level_2: str
    level_3: str
    level_4: str

ASSESSMENT_QUESTIONS = [
    AssessmentQuestion(
        dimension="Data Management",
        question="How do you manage training data?",
        level_0="Ad-hoc file storage",
        level_1="Versioned with DVC/Git LFS",
        level_2="Data validation in place",
        level_3="Feature store implemented",
        level_4="Automated data quality monitoring"
    ),
    # Add more questions for each dimension
]

def calculate_maturity_score(responses: List[int]) -> tuple:
    """Calculate overall maturity score."""
    avg_score = sum(responses) / len(responses)
    level = MaturityLevel(int(avg_score))
    return avg_score, level

Troubleshooting Guide

Common Issues

Debug Checklist

□ 1. Verify MLflow server is running
□ 2. Check tracking URI configuration
□ 3. Confirm artifact storage accessible
□ 4. Validate experiment exists
□ 5. Test with minimal example first

Knowledge Check

Self-Assessment Questions

1. What are the 5 main phases of the ML lifecycle?
2. How does MLOps differ from DevOps?
3. When would you use MLflow vs Weights & Biases?
4. What's the difference between Level 1 and Level 2 MLOps maturity?
5. What should you version in an ML project?

Practical Exercises

1. Beginner: Set up MLflow tracking for an existing notebook
2. Intermediate: Create a reproducible training script with environment capture
3. Advanced: Design an end-to-end pipeline diagram for a real use case

Resources

Official Documentation

• MLflow Documentation
• Google MLOps Guide

Recommended Reading

• "Designing Machine Learning Systems" by Chip Huyen
• "Machine Learning Engineering" by Andriy Burkov

Related Skills

• [See: experiment-tracking] - Deep dive into experiment tracking
• [See: training-pipelines] - Build production training pipelines
• [See: ml-infrastructure] - Set up ML infrastructure

Skill Completion Criteria

To mark this skill as complete, you must:

• Complete all 4 modules
• Finish at least 80% of exercises
• Pass the knowledge check (4/5 correct)
• Complete one practical project

Version History