Skip to main content
AmigoFish

Looped LLM? Recurrent Depth Transformer Explained

From bycloud · · 19 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.

If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.

Watch on YouTube

Not rated yet

Description

Need to fine-tune a model without the hassle? Try out Crusoe's serverless fine-tuning today! https://www.crusoe.ai/contact-sales/serverless-preview?utm_source=bycloud&utm_medium=influencer&utm_campaign=serverlessfinetuning

my latest project: Intuitive AI Academy We just wrote a new piece on Optimization!! https://intuitiveai.academy/ limited time code "LOCKIN" for 35% off yearly plan

Chain-of-Thought is ugly, so what if we remove it? In this video, I am diving into the latest architecture experiment that is Looped Transformer. So LLMs think with words, but here Looped Transformer simulates thinking by looping the same few layers.

My Newsletter https://mail.bycloud.ai/

My Patreon https://www.patreon.com/c/bycloud

Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers [Paper] https://arxiv.org/abs/2604.07822

A Mechanistic Analysis of Looped RLMs [Paper] https://arxiv.org/abs/2604.11791

Parcae: Scaling Laws For Stable Looped Language Models [Paper] https://arxiv.org/abs/2604.12946

Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation [Paper] https://arxiv.org/abs/2507.10524

Try out my new fav place to learn how to code https://scrimba.com/?via=bycloudAI

This video is supported by the kind Patrons & YouTube Members: 🙏Spam Maj, Alex, Chris LeDoux, DX Research Group, Poof N' Inu, Deagan, Robert Zawiasa, Ryszard Warzocha, Tobe2d, Louis Muk, Akkusativ, Kevin Tai, Mark Buckler, NO U, Tony Jimenez, Ângelo Fonseca, jiye, Anushka, Asad Dhamani, Binnie Yiu, Calvin Yan, Clayton Ford, Diego Silva, Etrotta, Gonzalo Fidalgo, Handenon, Hector, Jake Disco very, Michael Brenner, Nilly K, OlegWock, Daddy Wen, Shuhong Chen, Sid_Cipher, Stefan Lorenz, Sup, tantan assawade, Thipok Tham, Thomas Di Martino, Thomas Lin, Richárd Nagyfi, Paperboy, mika, Leo, Berhane-Meskel, Kadhai Pesalam, mayssam, Bill Mangrum, nyaa, Toru Mon, Lame Plane, Matej Macak, Len Mo, saylikhapekar, ZyanSheep, THEVIERAOS, Ricardo Raphael Corona-Moreno, superchordate

[Discord] https://discord.gg/NhJZGtH [Twitter] https://twitter.com/bycloudai [Patreon] https://www.patreon.com/bycloud [Business Inquiries] bycloud@smoothmedia.co [Other Inquiries] bycloudai@gmail.com [Profile & Banner Art] https://twitter.com/pygm7 [Video Editor] @aduckchicken2 Manim Animations created with Manimate https://www.manimate.ai/ [Ko-fi] https://ko-fi.com/bycloudai

Back to bycloud · Report a problem

Looped LLM? Recurrent Depth Transformer Explained | bycloud | AmigoFish