))
The Illusion of Thinking in Large Reasoning Models (LRM)
Manage episode 489624016 series 3605659
Inhalt bereitgestellt von Kabir. Alle Podcast-Inhalte, einschließlich Episoden, Grafiken und Podcast-Beschreibungen, werden direkt von Kabir oder seinem Podcast-Plattformpartner hochgeladen und bereitgestellt. Wenn Sie glauben, dass jemand Ihr urheberrechtlich geschütztes Werk ohne Ihre Erlaubnis nutzt, können Sie dem hier beschriebenen Verfahren folgen https://de.player.fm/legal.
This episode investigates the reasoning capabilities of Large Reasoning Models (LRMs), a new generation of language models designed for complex problem-solving. The authors evaluate LRMs using controllable puzzle environments to systematically analyze how performance changes with problem complexity, unlike traditional benchmarks that often suffer from data contamination. Key findings reveal three performance regimes: standard LLMs surprisingly excel at low complexity, LRMs gain an advantage at medium complexity, and both models experience complete collapse at high complexity, often exhibiting a counter-intuitive decline in reasoning effort despite having a sufficient token budget. The analysis also examines the internal reasoning traces, uncovering patterns like "overthinking" on simpler tasks and highlighting limitations in LRMs' ability to follow explicit algorithms or maintain consistent reasoning across different puzzle types.
Podcast:
https://kabir.buzzsprout.com
YouTube:
https://www.youtube.com/@kabirtechdives
Please subscribe and share.
322 Episoden
Teilen
