I Tried Coding a Neural Network to Identify Doodles
From Sebastian Lague · · 54 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.
If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.
Description
Exploring how neural networks learn by programming one from scratch in C#, and then attempting to teach it to recognize various doodles and images.
Support my work (and get early access to new videos and source code) on Patreon or Nebula * Patreon: https://www.patreon.com/SebastianLague * Nebula: https://go.nebula.tv/sebastianlague
Source code and Demo: * GitHub: https://github.com/SebLague/Neural-Network-Experiments * Demo: https://sebastian.itch.io/neural-network-experiment
Training Data: * Digits: http://yann.lecun.com/exdb/mnist/ * Fashion: https://github.com/zalandoresearch/fashion-mnist * Doodles: https://github.com/googlecreativelab/quickdraw-dataset * Cifar10: https://www.cs.toronto.edu/~kriz/cifar.html
Chapters 0:00 Introduction 2:39 The decision boundary 3:49 Weights 5:42 Biases 6:45 Hidden layers 7:45 Programming the network 9:57 Activation functions 12:42 Cost 15:07 Gradient descent example 18:22 The cost landscape 19:55 Programming gradient descent 21:10 It's learning! (slowly) 23:21 Calculus example 27:34 The chain rule 29:50 Some partial derivatives 33:14 Backpropagation 39:25 Digit recognition 43:56 Drawing our own digits 47:37 Fashion 48:25 Doodles 52:00 The final challenge
Music: Cosmic Waves - Michael FK Amber - The Stolen Orchestra Beyond the Horizon - Sounds Like Sander Air - Assaf Ayalon Purest Form - Sounds Like Sander Hear Wide Open - Sounds Like Sander Universal Wonder - Moments Roman P - Moments All In Good Time - Shimmer It Will Come Back - The Stolen Orchestra Frontier - Shimmer New Moon - Cloud Wave Sunflower - Cody Martin Inner Peace - Moments Enchanted - Cody Martin Just Around The Corner - Shimmer