Skip to main content
AmigoFish

The matrix math behind transformer neural networks, one step at a time!!!

From StatQuest with Josh Starmer · · 23 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.

If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.

Watch on YouTube

Not rated yet

Description

Transformers, the neural network architecture behind ChatGPT, do a lot of math. However, this math can be done quickly using matrix math because GPUs are optimized for it. Matrix math is also used when we code neural networks, so learning how ChatGPT does it will help you code your own. Thus, in this video, we go through the math one step at a time and explain what each step does so that you can use it on your own with confidence.

NOTE: This StatQuest assumes that you are already familiar with: Transformers: https://youtu.be/zxQyTK8quyY The essential matrix algebra for neural networks: https://youtu.be/bQ5BoolX9Ag

For a complete index of all the StatQuest videos, check out: https://statquest.org/video-index/

If you'd like to support StatQuest, please consider...

Patreon: https://www.patreon.com/statquest ...or... YouTube Membership: https://www.youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join

...buying one of my books, a study guide, a t-shirt or hoodie, or a song from the StatQuest store... https://statquest.org/statquest-store/

...or just donating to StatQuest! https://www.paypal.me/statquest

Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter: https://twitter.com/joshuastarmer

0:00 Awesome song and introduction 1:43 Word Embedding 3:37 Position Encoding 4:28 Self Attention 12:09 Residual Connections 13:08 Decoder Word Embedding and Position Encoding 15:33 Masked Self Attention 20:18 Encoder-Decoder Attention 21:31 Fully Connected Layer 22:16 SoftMax

#StatQuest #Transformer #ChatGPT

Back to StatQuest with Josh Starmer · Report a problem