Are Local Models Finally Good Enough?
From Techno Tim · · 25 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.
If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.
Description
I have been covering local and self-hosted AI for a few years now - from running models privately at home to what is still running in my homelab today. But to run the larger, more capable models, you need something more specialized than a general-purpose home server with a video card.
I wanted to see how good local self-hosted AI has actually gotten, so I built a mini AI cluster on my desk and tested it with a real coding workflow.
Can a local AI coding agent actually build and improve a real app? And does adding a second machine make the workflow better, or just add more complexity?
This is a practical test of the ASUS Ascent GX10, vLLM, OpenCode, power draw, Open Weight Models, and what local AI feels like to actually use.
Thank you to ASUS for sending these 2 GX10s to test. ASUS Ascent GX10: https://amzn.to/3PxWqjl (Amazon affiliate link)
Video Notes: https://technotim.com/posts/local-ai-gx10/
00:00 Local AI is starting to change 01:19 ASUS Ascent GX10 hardware 03:21 Remote access and workflow 04:19 NAS storage and vLLM 05:49 OpenCode agent stack 06:57 The Homelab app test 08:06 One GX10 coding test 11:48 One-GX10 power draw and idle cost 12:34 What does a second GX10 change? 13:00 NCCL and the two-node Ray cluster 14:09 MiniMax, KV cache, and model loading 16:09 Two GX10 coding test 17:44 Two-GX10 power draw 18:45 One GX10 vs two GX10s 19:12 UI and Windows XP stress tests 20:59 This is not plug and play 22:15 Final thoughts
Merch Shop 🛍️: https://l.technotim.com/shop Support me on Patreon: https://www.patreon.com/technotim Sponsor me on GitHub: https://github.com/sponsors/timothystewart6 Subscribe on Twitch: https://www.twitch.tv/technotim Become a YouTube member: https://www.youtube.com/channel/UCOk-gHyjcWZNj3Br4oxwh0A/join Gear Recommendations: https://l.technotim.com/gear Get Help in Our Discord Community: https://l.technotim.com/discord 2nd channel: https://www.youtube.com/@TechnoTimTinkers
(Affiliate links may be included in this description. I may receive a small commission at no cost to you.)
00:00 - Intro
Thank you for watching!