RLHF (Reinforcement Learning from Human Feedback)
Definition pending
We've cataloged "RLHF (Reinforcement Learning from Human Feedback)" but haven't written a full definition yet. Definitions are hand-curated rather than auto-generated, so it takes time to cover every term.
Want this one prioritized? Email us and we'll bump it.
Practical example
RLHF (Reinforcement Learning from Human Feedback) is the three-stage process: (1) supervised fine-tuning on demonstrations, (2) train a reward model on human preference comparisons, (3) optimize the policy with PPO against the reward model. It's the gold standard for alignment but operationally expensive: you need a team of labelers, a reward model training pipeline, and careful RL monitoring.
Workflow example
RLHF at scale: (1) collect 10K–100K human preference labels (services like Scale AI or Surge AI), (2) train reward model (typically a smaller model, 1–7B params), (3) run PPO with the reward model — this is the compute-heavy part, requiring 4–8 GPUs for a 7B policy model, (4) monitor reward score and KL divergence from base policy per training step, (5) if reward increases but KL diverges, you're "reward hacking" — the model found an exploit in your reward model. Most teams now default to DPO.