loading
loading
Reinforcement Learning from Human Feedback — training a model to produce answers humans prefer by optimizing against a learned reward signal.
Humans rank multiple model answers to the same prompt; those rankings train a separate reward model that scores responses, and then RL (usually PPO) updates the main model to maximize that score. It's the technique that made models feel aligned and conversational instead of merely fluent. The catch: the policy can learn to game the reward model (reward hacking), so you need a strong reward model and guardrails like a KL penalty to keep it from drifting off the rails.
Plainly
Think of RLHF as a simple recipe for doing the work better. Reinforcement Learning from Human Feedback — training a model to produce answers humans prefer by optimizing against a learned reward signal.
In practice
Use it when you need a repeatable method instead of guessing from vibes. In practice, define the owner, input, output, and failure mode before you rely on it.