People often dismiss LLMs as ‘just next-token predictors,’ but that’s way too simplistic. This overlooks the complex and unexpected behaviours that come out of the process. Tlaie Boria dives into how prediction, the core mechanism of LLMs, can actually lead to crazy levels of intelligence, reasoning, and creativity. By comparing this to evolution and human cognition, it shows that these models are capable of way more than just spitting out the next word—they’re building internal representations and capabilities that go far beyond basic word prediction.
Calling LLMs “just next-token predictors” is a bit like calling humans “just gene replication machines.” Both statements are superficially correct but profoundly miss the point. The objective that drives a system’s formation—whether it’s evolutionary fitness or next-token prediction—does not necessarily limit or fully describe the internal mechanics and emergent complexities that arise in pursuit of that objective.