2026
Breaking the Factorization Barrier in Diffusion Language Models
Ian Li, Zilei Shao, Benjie Wang, Rose Yu, Guy Van den Broeck, Anji Liu
ICML. 2026
We propose Coupled Discrete Diffusion (CoDD), a hybrid framework that breaks this barrier by replacing the fully-factorized output distribution with a lightweight, tractable probabilistic inference layer.
Breaking the Factorization Barrier in Diffusion Language Models
Ian Li, Zilei Shao, Benjie Wang, Rose Yu, Guy Van den Broeck, Anji Liu
ICML. 2026
We propose Coupled Discrete Diffusion (CoDD), a hybrid framework that breaks this barrier by replacing the fully-factorized output distribution with a lightweight, tractable probabilistic inference layer.
Manifold-Guided Attention Steering
Ian Li, Kapilesh Guruprasad, Raunak Sengupta, Ninad Satish, Loris D'Antoni, Rose Yu
Under Review. 2026
We propose Manifold-Guided Attention Steering (MAGS), a trajectory-aware inference-time intervention grounded in a geometric observation that the output activations of specific attention heads diverge from a low-dimensional correctness manifold at the point of error in reasoning problems.
Manifold-Guided Attention Steering
Ian Li, Kapilesh Guruprasad, Raunak Sengupta, Ninad Satish, Loris D'Antoni, Rose Yu
Under Review. 2026
We propose Manifold-Guided Attention Steering (MAGS), a trajectory-aware inference-time intervention grounded in a geometric observation that the output activations of specific attention heads diverge from a low-dimensional correctness manifold at the point of error in reasoning problems.
2025
Learning Tractable Distributions of Language Model Continuations
Gwen Yidou-Weng, Ian Li, Anji Liu, Oliver Broadrick, Guy Van den Broeck, Benjie Wang
ArXiv 2025
We propose Learning to Look Ahead (LTLA), a hybrid approach that pairs the same base language model for rich prefix encoding with a fixed tractable surrogate model that computes exact continuation probabilities.
Learning Tractable Distributions of Language Model Continuations
Gwen Yidou-Weng, Ian Li, Anji Liu, Oliver Broadrick, Guy Van den Broeck, Benjie Wang
ArXiv 2025
We propose Learning to Look Ahead (LTLA), a hybrid approach that pairs the same base language model for rich prefix encoding with a fixed tractable surrogate model that computes exact continuation probabilities.
Steering LLMs’ Reasoning With Activation State Machines
Ian Li, Philip Chen, Max Huang, Andrew Park, Loris D'Antoni, Rose Yu
FoRLM @ NeurIPS 2025 2025
We introduce Activation State Machine (ASM), an lightweight dynamic steering mechanism that learns the latent dynamics of ideal reasoning trajectories and applies context-aware interventions at inference time.
Steering LLMs’ Reasoning With Activation State Machines
Ian Li, Philip Chen, Max Huang, Andrew Park, Loris D'Antoni, Rose Yu
FoRLM @ NeurIPS 2025 2025
We introduce Activation State Machine (ASM), an lightweight dynamic steering mechanism that learns the latent dynamics of ideal reasoning trajectories and applies context-aware interventions at inference time.