Differential Privacy

Differential Privacy is a mathematical framework that provides a formal guarantee that the output of an analysis reveals almost nothing about whether any individual's data was included in the input. A randomized mechanism M satisfies (ε, δ)-differential privacy if for all datasets D, D′ differing by one record and all output sets S: P(M(D) ∈ S) ≤ eᵋ P(M(D′) ∈ S) + δ. The privacy parameter ε (0.1-8.0) quantifies the privacy loss — lower ε means stronger privacy but degraded utility. The Gaussian mechanism adds calibrated noise σ√(2log(1.25/δ))/ε to each query output; for ε = 1, training an ML model on 100K records requires adding noise with σ proportional to 1/1 = 1× the query sensitivity.

Differential privacy (DP) adds calibrated noise to data or model outputs so that the presence or absence of any individual can't be determined. It provides a mathematical guarantee: ε (epsilon) quantifies the privacy loss — lower ε = stronger privacy. For operators, DP protects training data from being extracted from the model.

Differential privacy deployment: (1) for training: use DP-SGD (differentially private stochastic gradient descent) — adds noise to gradients, (2) ε budget: total privacy loss allowed across all queries — 1.0 is strong privacy, 10 is moderate, 100+ is weak, (3) trade-off: stronger privacy (lower ε) = lower model accuracy, (4) for LLM fine-tuning: DP fine-tuning protects sensitive training data from extraction attacks, (5) for production: DP is used when training data contains personal information — medical records, financial data, private messages.