Skip to main content
AmigoFish

Apple didn't have to go this hard...

From Jeff Geerling · · 14 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.

If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.

Watch on YouTube

Not rated yet

Description

RDMA over Thunderbolt support landed in macOS 26.2.

Is it good for HPC? For Exo and LLMs? Find out in this video.

Resources (some links are affiliate links):

- Mac Studio: https://www.apple.com/mac-studio/ - Exo 1.0: https://exolabs.net - Apple MLX PR for Thunderbolt RDMA: https://github.com/ml-explore/mlx/pull/2808 - M5 Neural Accelerator speedup: https://machinelearning.apple.com/research/exploring-llms-mlx-m5 - DeskPi Rackmate TL1: https://amzn.to/4p5ek94 - Tupavco mini-rack PDU: https://amzn.to/4j0DpAq

Disclaimer: The Mac Studios used in this cluster are on loan from Apple for testing. They are not sponsoring this video, nor had any say in the contents. DeskPi sent the Rackmate TL1 used in the video for my testing, but they also had no say in the video contents. I always add the "sponsored" flag on my videos if I did not purchase the equipment being shown.

Support me on Patreon: https://www.patreon.com/geerlingguy Sponsor me on GitHub: https://github.com/sponsors/geerlingguy Merch: https://www.redshirtjeff.com 2nd Channel: https://www.youtube.com/@GeerlingEngineering 3rd Channel: https://www.youtube.com/@Level2Jeff

Contents:

00:00 - A rack-full of Macs 02:01 - Putting together a cluster 03:16 - The downsides of Thunderbolt 05:09 - One is enough 07:45 - Managing macOS in a cluster (vs Linux) 08:37 - HPL and llama.cpp cluster performance 09:30 - Enabling RDMA 09:51 - Massive LLMs 11:34 - RDMA over TB5 is great... when it works 12:10 - EXO community and OSS 12:51 - Apple could go harder 13:30 - Jack of all trades, not just AI

Back to Jeff Geerling · Report a problem