Ax Mingzhi Chen, Taiming Lu, Jiachen Zhu, Mingjie Sun, Zhuang Liu 4/1/2026

Stronger Normalization-Free Transformers

Research on normalization-free transformer architectures using Dynamic Tanh as alternative to standard normalization layers.

Ax Yuze Wang, Yujia Tong, Xuan Liu, Junhao Dong 4/1/2026

Sparsity-Aware Unlearning for Large Language Models

Addresses machine unlearning for sparse LLMs to remove memorized sensitive information while maintaining model sparsification benefits for efficient deployment.

Ax Jie Xiao, Meng Chen, Qingnan Ren, Jingwei Song, Jiaqi Huang, Yangshen Deng, Chris Tong, Wanyi Chen, Suli Wang, Ziqian Bi, Shuo Lu, Yiqun Duan, Xu Wang, Rymon Yu, Ween Yang, Lynn Ai, Eric Yang, Bill Shi 4/1/2026

ECHO-2: A Large-Scale Distributed Rollout Framework for Cost-Efficient Reinforcement Learning

ECHO-2 is distributed RL framework for LLM post-training via reinforcement learning, optimizing cost-efficiency of rollout generation across distributed resources.

Ax Amin Oji, Paul Fieguth 4/1/2026

Joint Embedding Variational Bayes

VJE introduces reconstruction-free latent-variable framework for self-supervised learning using symmetric conditional ELBO on paired embeddings.

Ax YanZhao Zheng, ZhenTao Zhang, Chao Ma, YuanQiang Yu, JiHuai Zhu, Yong Wu, Tianze Xu, Baohua Dong, Hangcheng Zhu, Ruohui Huang, Gang Yu 4/1/2026

SkillRouter: Skill Routing for LLM Agents at Scale

SkillRouter: System for routing LLM agent requests to relevant skills from large skill libraries at inference time.

Ax Dong Zhuo, Wenzhao Zheng, Jiahe Guo, Yuqi Wu, Jie Zhou, Jiwen Lu 4/1/2026

Streaming 4D Visual Geometry Transformer

Streaming transformer architecture inspired by autoregressive LLMs for real-time 3D geometry perception and reconstruction from video.

Ax Yixuan Wang, Huang He, Siqi Bao, Hua Wu, Haifeng Wang, Qingfu Zhu, Wanxiang Che 4/1/2026

ProxyAttn: Guided Sparse Attention via Representative Heads

ProxyAttn method using representative attention heads to enable efficient sparse attention in LLMs for long-text processing with minimal performance degradation.

Ax Yi-Kai Zhang, Zhiyuan Yao, Hongyan Hao, Yueqing Sun, Qi Gu, Hui Su, Xunliang Cai, De-Chuan Zhan, Han-Jia Ye 4/1/2026

$V_0$: A Generalist Value Model for Any Policy at State Zero

V0 is a generalist value model for policy gradient methods that scales efficiently with LLM training, replacing large critic models in actor-critic methods like PPO.