MLflow

MLflow is an open-source platform for managing the end-to-end machine learning lifecycle, including experimentation, reproducibility, and deployment. Operators encounter it when tracking model training runs, comparing hyperparameters and metrics, or packaging models for serving. It provides four main components: Tracking (logging parameters, metrics, artifacts), Projects (packaging code for reproducibility), Models (standard format for model storage and serving), and Registry (centralized model versioning). For local AI operators, MLflow is useful for organizing local experiments, especially when iterating on fine-tuning or evaluating multiple quantized models.

MLflow's Tracking component is most relevant to operators: it logs parameters (e.g., learning rate, quantization bits), metrics (e.g., perplexity, tokens/sec), and artifacts (e.g., model weights, plots) to a local or remote server. Operators can run mlflow ui to view runs in a browser, comparing experiments side-by-side. The Models component defines a standard packaging format (MLflow Model) that can be served via REST API using mlflow models serve. For local AI, this is useful when deploying a fine-tuned model as a local inference server. The Registry allows versioning and stage transitions (Staging, Production) for models, though this is more relevant in team settings. MLflow integrates with many ML frameworks, including Hugging Face Transformers, PyTorch, and TensorFlow, making it a practical tool for tracking local fine-tuning jobs.

An operator fine-tunes Llama 3.1 8B on a custom dataset using Hugging Face Transformers. They use MLflow Tracking to log hyperparameters (learning rate, batch size, quantization type), metrics (training loss, validation perplexity), and artifacts (the final Q4_K_M GGUF file). After training, they run mlflow ui to compare this run against previous fine-tuning attempts, selecting the best model based on validation perplexity. They then package the model as an MLflow Model and serve it locally with mlflow models serve -m runs:/<run_id>/model --port 8080.