What Is an NPU Actually Good For?
From Techno Tim · · 13 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.
If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.
Description
What is an NPU actually useful for? Is it faster than a GPU, more efficient, or meant for a completely different kind of AI workload?
I tested the Intel NPU in the MINISFORUM MS-03 using Frigate and OpenVINO, comparing the same workload across the CPU, GPU, and NPU.
MINISFORUM sent the MS-03 for me to test and that's why you see the disclosure overlay. This video is not sponsored, they had no input into this video.
Video Notes: https://technotim.com/posts/intel-npu-frigate/
NON-AFFILIATE LINK - MINISFORUM MS-03: https://store.minisforum.com/products/minisforum-ms-03-workstation
Amazon AFFILIATE LINKS - Intel CPUs with NPU: https://amzn.to/4zyzO4r - AMD NPU laptops (no desktop CPUs with NPUs): https://amzn.to/4ghirw8 - Apple devices with NPU: https://amzn.to/3S1BG56 - Google Pixel with NPU: https://amzn.to/3Sm6IVo - MINISFORUM MS-03: https://amzn.to/3UOCCdN
Merch Shop 🛍️: https://l.technotim.com/shop Support me on Patreon: https://www.patreon.com/technotim Sponsor me on GitHub: https://github.com/sponsors/timothystewart6 Subscribe on Twitch: https://www.twitch.tv/technotim Become a YouTube member: https://www.youtube.com/channel/UCOk-gHyjcWZNj3Br4oxwh0A/join Gear Recommendations: https://l.technotim.com/gear Get Help in Our Discord Community: https://l.technotim.com/discord 2nd channel: https://www.youtube.com/@TechnoTimTinkers
(Affiliate links may be included in this description. I may receive a small commission at no cost to you.)
00:00 The Third Processor 00:40 What Exactly Is an NPU? 01:33 Finding a Real NPU Workload 03:01 Meet the MINISFORUM MS-03 04:30 When Panther Lake Is Too New 05:14 Giving Frigate a Real Job 07:19 Making CPU vs GPU vs NPU Fair 08:31 The Moment of Truth 09:56 Why the Benchmarks Disagree 11:31 Finding the NPU's Job 12:17 LLMs and the Workloads I Left Out
Thank you for watching!