Ax Akshay Rangamani, Altay Unal 3/26/2026

Deep Neural Regression Collapse

Neural Regression Collapse phenomenon across network layers showing feature sparsity and low rank structures.

Ax Guoliang Zhao, Ruobing Xie, An Wang, Shuaipeng Li, Huaibing Xie, Xingwu Sun 3/26/2026

Self-Distillation for Multi-Token Prediction

Self-distillation method for multi-token prediction in LLMs to improve inference efficiency and MTP head acceptance rates.

Ax Allen Nie, Xavier Daull, Zhiyi Kuang, Abhinav Akkiraju, Anish Chaudhuri, Max Piasevoli, Ryan Rong, YuCheng Yuan, Prerit Choudhary, Shannon Xiao, Rasool Fakoor, Adith Swaminathan, Ching-An Cheng 3/26/2026

Understanding the Challenges in Iterative Generative Optimization with LLMs

Analysis of challenges in iterative generative optimization with LLMs for self-improving agents, identifying hidden design choices limiting adoption.