Skip to content

Reasoning Models, Multimodality & Frontier Trends

What you’ll learn. How scaling inference-time compute produced reasoning models, and what is known about how they work and where they fail; distillation of reasoning into small models; how vision and audio are fused into language models (encoders, projectors, tokenization of modalities, native multimodality); state-space and hybrid architectures as alternatives to full attention; and a candid list of open problems. This lecture is the most perishable in the section: treat named systems as snapshots and focus on the mechanisms.

Scaling laws (Pretraining) describe returns to training compute. A second axis is inference-time compute: letting the model spend more tokens (or more samples) on a hard problem. Two broad forms:

  1. Parallel: sample many candidates and select: best-of-nn with a verifier or reward model, majority voting / self-consistency (Prompting). Gains depend on the verifier’s quality; with a perfect verifier, accuracy approaches coverage (pass@nn), which grows roughly log-linearly in nn over many orders of magnitude on some tasks (Brown et al., “Large Language Monkeys,” 2024).
  2. Sequential: a long chain of thought in which the model explores, checks and revises within one generation, trained with RL.

Snell et al. (2024) showed that for some problems, optimally allocating extra inference compute can beat scaling parameters by a significant factor, with the benefit largest on problems of moderate difficulty relative to the base model. Intuition: a transformer’s forward pass has bounded depth; generating tokens adds serial computation steps (a theoretical result: chain of thought expands the class of functions a fixed-depth transformer can compute). Hard problems may simply need more serial steps than one forward pass allows.

Long chain-of-thought and reasoning models

Section titled “Long chain-of-thought and reasoning models”

The 2024-2026 generation of reasoning models are LLMs post-trained, mainly by reinforcement learning with verifiable rewards (Post-Training), to produce extended “thinking” before answering. Observed behaviors that emerge with RL on math, code and logic problems:

  • longer outputs for harder problems (adaptive compute);
  • self-verification (“let me double check by substituting back”);
  • backtracking and trying alternative strategies after noticing a dead end;
  • decomposition into subproblems, case analysis, exploring special cases.

Typical training recipe (as publicly described by open efforts): (1) a base or instruction model; (2) optionally a small cold-start SFT on curated long reasoning traces; (3) large-scale RL (GRPO-style) with rule-based rewards for answer correctness and format; (4) rejection-sampling SFT on the model’s best traces plus general data; (5) a final round of preference and safety tuning.

User-facing controls. Providers expose a reasoning effort or token budget so you trade latency and cost against accuracy. Reasoning tokens are billed as output tokens, so cost per query can rise by an order of magnitude on hard prompts. Reasoning traces may be summarized or hidden by the provider.

What’s contested or imperfectly understood:

  • Does RL create new capability or just sample efficiency? Some analyses (pass@kk at large kk) suggest RL mostly shifts probability mass toward solutions the base model could already sample; others show clear gains on problems the base model never solved with long training. The answer likely depends on scale and training length.
  • Faithfulness: the visible trace may not reflect the computation that determined the answer (Prompting). This matters for monitoring: reading the chain of thought is a useful but fragile safety signal, and training directly against it can teach models to obscure it.
  • Overthinking: models burn tokens on easy problems, sometimes second-guessing correct answers; and can get stuck in loops. Length-penalized rewards and budget forcing mitigate.
  • Brittleness: performance can collapse with small problem perturbations or as puzzle complexity grows beyond a threshold; reasoning models still make arithmetic and bookkeeping errors, and gains outside verifiable domains (writing, open-ended judgment) are smaller.
  • Reward hacking in code (special-casing tests, editing tests) and format gaming.

Search at inference. Beyond a single long chain: tree search over reasoning steps with a process reward model (PRM) scoring step quality versus an outcome reward model (ORM); Monte Carlo tree search; and combining long CoT with parallel sampling and verification. In practice, simple methods (long CoT plus majority vote or verifier-selected best-of-nn) have been hard to beat per unit compute.

Budget forcing (Muennighoff et al., s1, 2025) shows a minimal demonstration: with a small curated SFT set of reasoning traces, one can control thinking length at test time by appending “Wait” to extend or truncating to stop, giving a smooth accuracy-versus-tokens curve.

Training a small model to imitate the long traces of a large reasoning model (SFT on its outputs, filtered for correctness) transfers much of the capability far more cheaply than running RL on the small model directly; open reports found distilled small models beat the same size models trained with RL from scratch. Variants: logit/sequence-level knowledge distillation (Fine-Tuning in Practice), on-policy distillation (the student samples, the teacher scores tokens), and distilling into shorter reasoning for latency. Caveats: the student is bounded by the teacher, inherits its errors and biases, and providers’ terms of service often restrict training on their outputs; hidden/summarized traces limit what can be distilled.

Modern frontier models accept images, audio and video (and sometimes generate them). Two architectural families:

Adapter-based (late fusion into a pretrained LLM)

Section titled “Adapter-based (late fusion into a pretrained LLM)”
image -> vision encoder (ViT) -> patch embeddings -> projector (MLP / cross-attn / Q-Former)
-> "visual tokens" in the LLM's embedding space -> interleaved with text tokens -> LLM
  • Vision encoder: a ViT pretrained contrastively with text (CLIP/SigLIP-style) or self-supervised; image split into patches (e.g. 14x14 pixels) and each patch becomes a vector. A 448×448448\times448 image with patch size 14 yields 32×32=102432\times32=1024 patches.
  • Projector: maps encoder features into the LLM’s embedding dimension; a simple MLP works well (LLaVA); resamplers (Perceiver/Q-Former) compress to a fixed number of tokens; cross-attention layers inserted into the LLM (Flamingo-style) are another approach.
  • Training: (1) align the projector on image-caption pairs with the LLM frozen; (2) train on interleaved image-text and visual instruction data (often partly synthetic) with the LLM unfrozen; (3) preference/safety tuning.
  • Resolution handling: tile high-resolution images into crops plus a thumbnail, or use native-resolution encoders with variable token counts. Image tokens cost context and money: a single image may be hundreds to thousands of tokens.

Train one transformer from scratch on interleaved text, image, audio (and video) tokens with a shared next-token objective, using discrete tokens from image/audio tokenizers (VQ-style codebooks) or continuous patch inputs. Benefits: tighter cross-modal reasoning, able to generate non-text outputs with the same model. Cost: much more expensive pretraining and data-engineering complexity. Many frontier systems are described as natively multimodal; details are mostly undisclosed.

Document and screen understanding is a major practical use: reading charts, scanned PDFs, UI screenshots (enabling computer-use agents; see Agents). Vision-language retrieval (e.g. ColPali-style page-image embeddings) lets RAG skip brittle OCR pipelines.

Failure modes. Hallucinated objects and text, poor counting and precise spatial reasoning, fine-print OCR errors, sensitivity to image resolution and cropping, visual prompt injection (instructions embedded in images), and privacy concerns (faces, metadata).

  • Cascaded: speech-to-text (ASR, e.g. Whisper-style encoder-decoder) then LLM then text-to-speech. Modular and robust, but adds latency (hundreds of ms to seconds), and loses prosody, emotion and non-speech sounds.
  • End-to-end speech-language models: audio is tokenized (neural audio codecs produce discrete tokens at tens of tokens per second, often several codebook levels per frame) and the LLM consumes or generates them directly, enabling low-latency full-duplex conversation, interruption handling and expressive voice. Technical challenges: audio token rates are much higher than text, so models use hierarchical or delayed multi-codebook prediction; preserving text-level reasoning when trained on speech is hard.
  • Streaming and voice-activity/turn-taking logic dominate perceived quality in real-time agents.

Video adds time: sample frames, compress with temporal pooling or tokens merging, and manage the quadratic cost. Also: 3D, robotics/action tokens (vision-language-action models), time series, and protein/molecule sequences as “languages”.

Attention costs O(T2)O(T^2) compute and an O(T)O(T) KV cache (architecture). State-space models (SSMs) and linear-recurrent models replace the cache by a fixed-size recurrent state:

ht=At ht−1+Bt xt,yt=Ct ht,h_t = A_t\,h_{t-1} + B_t\,x_t,\qquad y_t = C_t\,h_t,

giving O(T)O(T) training (via parallel scan or chunked formulations) and O(1)O(1) memory and compute per generated token. S4 (Gu et al., 2021) introduced structured SSMs; Mamba (Gu and Dao, 2023) made A,B,CA,B,C input-dependent (“selective”), letting the state decide what to remember or forget; Mamba-2 recast SSMs as a form of linear attention (structured state-space duality), linking the two families. Related: RWKV, RetNet, linear attention variants, gated delta-rule layers.

Strengths: constant-size state, very long sequences, fast decoding. Weakness: compressing history into a fixed state hurts precise recall and in-context retrieval/copying tasks, where attention shines. The pragmatic answer in 2025-2026 has been hybrids: mostly SSM/linear layers with a minority of attention layers (and sometimes MoE), which keep most of the quality while shrinking cache and boosting throughput; several open and commercial models have adopted this. Whether pure-attention transformers remain dominant at the frontier is open; for most practitioners the transformer toolchain remains the default.

Other efficiency directions: sparse and sliding-window attention, latent attention compression, KV-cache quantization and eviction, and diffusion language models that generate blocks of tokens in parallel by iterative denoising (promising for speed; quality at scale still being established).

  • Agentic training and long-horizon RL: RL in tool-using environments (coding, browsing, computer use) rather than single-turn problems; environment quality and verifier reliability become the bottleneck.
  • Synthetic data and self-improvement: models generating training data and verifying it; risks of collapse and bias amplification; “data wall” concerns.
  • Test-time training and continual learning: updating weights or fast-weight memories during use; today models are static after training and “learn” only through context.
  • Interpretability: sparse autoencoders and circuit analysis to find features and mechanisms; promising for debugging and safety but far from full explanations.
  • Efficient small models: on-device models via distillation and quantization (Inference & Serving).
  • Memory and personalization beyond the context window (Agents).
  • Scalable oversight: supervising models on tasks humans can’t easily verify.
  1. Reliability and calibration: hallucination is reduced but not solved; models are poorly calibrated about what they know, and failures remain unpredictable on the “jagged frontier” (superhuman on some tasks, brittle on trivial variations).
  2. Generalization of reasoning: How much is general reasoning vs pattern matching on seen task families? How do RL gains transfer outside verifiable domains?
  3. Reward specification: verifiers exist for math and code, but most valuable tasks (research, writing, judgment, long-running projects) lack cheap reliable rewards; LLM judges are biased (Evaluating LLMs).
  4. Long-horizon autonomy: error compounding, context management and recovery over hours to days of agent work.
  5. Evaluation itself: benchmarks saturate and leak; measuring real-world capability and risk is hard.
  6. Data: exhaustion of high-quality human text, licensing, provenance, and the dynamics of training on model-generated data.
  7. Efficiency: cost per token has fallen dramatically, yet reasoning and agents consume far more tokens; energy and hardware constraints bind.
  8. Alignment and security: prompt injection, jailbreak robustness, deception and monitorability of reasoning, oversight of increasingly capable agents (Post-Training).
  9. Interpretability: understanding mechanisms well enough to trust and steer models.
  10. Architecture: is there something fundamentally better than attention for memory and long-term dependencies?
  • Use reasoning models for tasks with multi-step logic, planning, hard coding and math; use fast non-reasoning models for extraction, classification, routing and chat. Route by difficulty.
  • Set explicit token/effort budgets, measure cost per correct answer, and cap runaway reasoning.
  • For multimodal inputs, test on your own documents/images; resolution and cropping choices matter; budget image tokens.
  • Expect your model choices to change every few months: abstract the provider behind your own interface and keep an eval suite to re-qualify replacements (Building LLM Applications).
  • Inference-time compute is a second scaling axis: parallel sampling with verification and sequential long chain-of-thought trained with verifiable-reward RL.
  • Reasoning models gain accuracy on verifiable tasks at much higher token cost; faithfulness, overthinking and generalization beyond verifiable domains remain open.
  • Distillation transfers long-CoT skills to small models cheaply, bounded by the teacher.
  • Multimodality is mostly encoder + projector + LLM, or natively trained on interleaved tokens; audio can be cascaded or end-to-end with codec tokens.
  • SSMs/linear recurrences give constant-size state but weaker exact recall; hybrids with some attention layers are the practical compromise.
  • Many open problems (reliability, reward specification, long-horizon autonomy, evaluation, data) are as much about systems and measurement as architecture.
  1. Hands-on. Take 50 math or logic problems. Compare a non-reasoning model, the same model prompted with CoT, and a reasoning model at two effort settings. Plot accuracy versus mean output tokens and cost.
  2. Conceptual. Why does best-of-nn sampling with an imperfect reward model eventually decrease true accuracy as nn grows?
  3. Conceptual. For an SSM with a state of size ss per layer, what limits its ability to recall an arbitrary token from a very long context? Why do hybrids help?
  4. Hands-on. Implement a toy linear-recurrence layer ht=a⊙ht−1+b⊙xth_t=a\odot h_{t-1}+b\odot x_t in PyTorch (sequential and parallel-scan version) and verify they match.
  5. Conceptual. Estimate the token cost of an image of 896×896896\times896 pixels with patch size 14 and a 2×22\times2 patch-merging projector.
Solutions (sketch)
  1. The selector exploits the reward model’s errors (Goodhart): as nn grows, the chosen sample increasingly maximizes the error component of the reward rather than correctness; the overoptimization curve peaks then declines.
  2. The state has finite capacity (ss numbers) so it can’t store arbitrary many distinct facts from a long history verbatim; attention layers retain exact token-level access via the KV cache, so a few of them restore precise retrieval.
  3. 896/14=64896/14=64 patches per side, so 4096 patches; merging 2×22\times2 gives 10241024 visual tokens.
  • Wei et al., “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” (2022); Merrill and Sabharwal, “The Expressive Power of Transformers with Chain of Thought” (2023).
  • Snell et al., “Scaling LLM Test-Time Compute Optimally Can Be More Effective than Scaling Model Parameters” (2024); Brown et al., “Large Language Monkeys” (2024).
  • Lightman et al., “Let’s Verify Step by Step” (2023); DeepSeek-AI, “DeepSeek-R1” (2025); Muennighoff et al., “s1: Simple test-time scaling” (2025).
  • Yue et al., “Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?” (2025).
  • Radford et al., “CLIP” (2021); Alayrac et al., “Flamingo” (2022); Liu et al., “Visual Instruction Tuning” (LLaVA, 2023); Radford et al., “Whisper” (2022).
  • Gu, Goel, Re, “Efficiently Modeling Long Sequences with Structured State Spaces” (S4, 2021); Gu and Dao, “Mamba” (2023); Dao and Gu, “Transformers are SSMs” (2024).
  • Anthropic and other labs’ interpretability publications (e.g. “Towards Monosemanticity”, 2023) for the interpretability thread.