Skip to main content
AmigoFish

Reinforcement Learning with Human Feedback (RLHF), Clearly Explained!!!

From StatQuest with Josh Starmer · · 18 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.

If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.

Watch on YouTube

Not rated yet

Description

Generative Large Language Models, like ChatGPT and DeepSeek, are trained on massive text based datasets, like the entire Wikipedia. However, this training alone fails to teach the models how to generate polite and useful responses to your prompts. Thus, LLMs rely on Supervised Fine-Tuning and Reinforcement Learning with Human Feedback (RLHF) to align the models to how we actually want to use them. This StatQuest explains every step in training an LLM, with special attention to how RLHF is done.

NOTE: This video is based on the original manuscript for Instruct-GPT: https://arxiv.org/abs/2203.02155

Also, you should check out Serrano Academy if you can: https://www.youtube.com/@SerranoAcademy

If you'd like to support StatQuest, please consider... Patreon: https://www.patreon.com/statquest ...or... YouTube Membership: https://www.youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join

...buying a book, a study guide, a t-shirt or hoodie, or a song from the StatQuest store... https://statquest.org/statquest-store/

...or just donating to StatQuest! paypal: https://www.paypal.me/statquest venmo: @JoshStarmer

Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter: https://twitter.com/joshuastarmer

0:00 Awesome song and introduction 2:25 Pre-Training an LLM 5:06 Supervised Fine-Tuning 7:35 Reinforcement Learning with Human Feedback (RLHF) 10:07 RLHF - training the reward model 15:02 RLHF - using the reward model

#StatQuest

Back to StatQuest with Josh Starmer · Report a problem