Ax Neo Christopher Chung, Maxim Laletin 4/3/2026

Regularizing Attention Scores with Bootstrapping

Research on regularizing attention scores in vision transformers using bootstrapping to improve interpretability and reduce noisy attention maps.

Ax Zhehang Du, Weijie Su 4/3/2026

The Newton-Muon Optimizer

New optimizer deriving design principles from Muon, improving LLM training efficiency through surrogate model analysis.

Ax Ga\"etan Hadjeres, Marc Ferras, Khaled Koutini, Benno Weck, Alexandre Bittar, Thomas Hummel, Zineb Lahrici, Hakim Missoum, Joan Serr\`a, Yuki Mitsufuji 4/3/2026

Woosh: A Sound Effects Foundation Model

Woosh: open-source sound effects foundation model from Sony AI with architecture, training details, and benchmarks.

Ax Hugo Koubbi, Borjan Geshkovski, Philippe Rigollet 4/3/2026

Homogenized Transformers

Theoretical analysis of multi-head self-attention transformers using particle systems and homogenization limits.

Ax Atilla Kaan Alkan, Felix Grezes, Sergi Blanco-Cuaresma, Jennifer Lynn Bartlett, Daniel Chivvis, Anna Kelbert, Kelly Lockhart, Alberto Accomazzi 4/3/2026

AstroConcepts: A Large-Scale Multi-Label Classification Corpus for Astrophysics

AstroConcepts corpus of 21,702 astrophysics abstracts for multi-label classification research addressing extreme class imbalance with specialized terminology.

Ax Daiwei Chen, Zhoutong Fu, Chengming Jiang, Haichao Zhang, Ran Zhou, Tan Wang, Chunnan Yao, Guoyao Li, Rui Cai, Yihan Cao, Ruijie Jiang, Fedor Borisyuk, Jianqiang Shen, Jingwei Wu, Ramya Korlakai Vinayak 4/3/2026

Grounded Token Initialization for New Vocabulary in LMs for Generative Recommendation

Analysis of token initialization strategies for new vocabulary in language models used for generative recommendation systems.

Ax Dmitrii Krylov, Armin Karamzade, Roy Fox 4/3/2026

Moonwalk: Inverse-Forward Differentiation

Moonwalk: inverse-forward differentiation technique addressing backpropagation's memory requirements for training deeper networks.