MLOps

MLOps (Machine Learning Operations) covers the infrastructure, processes, and tooling needed to deploy, monitor, and maintain ML models in production. It draws from DevOps (CI/CD, infrastructure-as-code), data engineering (pipelines, feature management), and ML engineering (model versioning, experiment tracking, serving).

ML pipeline: the sequence from raw data → feature engineering → training → evaluation → deployment → monitoring. Each stage must be automated, versioned, and reproducible. A change to the data schema upstream should trigger retraining and evaluation before deployment.

Experiment tracking: every training run should record hyperparameters, metrics, dataset version, and code commit — enabling reproducibility and comparison. MLflow, Weights & Biases, and Neptune are common tools.

Feature store: a centralized repository of computed features shared across teams. Solves training-serving skew (the feature computation differs between training and production) and prevents redundant feature engineering across teams. Examples: Feast, Tecton, Hopsworks.

Model registry: version control for trained models — stores artifacts, metadata, and promotes models through staging → production. Prevents "which model is in production?" confusion.

CI/CD for ML: automated pipelines that test data quality, retrain models on new data, run evaluation checks, and deploy to serving infrastructure. A model should not reach production without passing evaluation thresholds.

Model drift: model performance degrades over time as the real world changes. Data drift: input distribution changes. Concept drift: the relationship between inputs and targets changes. Monitoring with statistical tests (PSI, KS test) or reference distributions detects drift and triggers retraining.

Model serving: REST/gRPC APIs, batch inference pipelines, or streaming inference. Latency and throughput are primary concerns. Model optimization (quantization, distillation, TensorRT) reduces serving cost.

A/B testing for models: canary rollouts, shadow mode (log predictions without serving them), and champion-challenger frameworks validate new models against production baselines before full traffic switch.

Most ML projects fail not because of algorithmic problems, but because of operational ones: models trained on stale data, features computed differently in training vs production, no monitoring until users complain, manual deployment steps that introduce errors. MLOps addresses the 80% of ML work that happens after the model is trained. A 3% model improvement that takes a month to deploy adds less value than a 1% model improvement that deploys daily and catches drift automatically.

ML Engineer and senior data science roles test MLOps heavily. Expect system design questions: "design a real-time fraud detection pipeline," "how would you retrain a recommendation model daily at scale," or "how would you detect when a model has degraded?" Knowing the failure modes (training-serving skew, silent drift, feature leakage in pipelines, deployment rollback triggers) demonstrates production ML experience that differentiates candidates. Entry-level roles test familiarity with the concepts; senior roles test the tradeoffs and war stories.