Ax Jack Young 14h ago

WriteSAE: Sparse Autoencoders for Recurrent State

WriteSAE introduces sparse autoencoders for matrix updates in recurrent language models like Mamba-2 and RWKV-7, learning rank-1 matrix atoms to directly replace model writes.

Ax Yuan Zhang, Lifeng Guo, Junwen Pan, Wenzhao Zheng, Wen Zhou, Kuan Cheng, Kurt Keutzer, Shanghang Zhang 14h ago

SEED: Targeted Data Selection by Weighted Independent Set

SEED: data selection method formulated as weighted independent set problem to balance quality and diversity in training datasets.

Ax Muhammad Umer, Muhammad Ahmed Mohsin, Ahsan Bilal, Arslan Chaudhry, Andreas Haupt, Sanmi Koyejo, Emily Fox, John M. Cioffi 14h ago

General Preference Reinforcement Learning

General Preference RL framework unifying online RL and preference optimization for LLM alignment across both structured and open-ended tasks.

Ax Weinuo Ou 14h ago

Exact Linear Attention

Exact Linear Attention mechanism achieving linear computational complexity for Transformers through kernel decomposition without approximation error.

Ax Antoine Boutet, Lucas Magnana, Juliette S\'en\'echal 14h ago

Towards the Anonymization of the Language Modeling

Privacy-preserving language modeling approach to prevent memorization and exposure of sensitive personal information in fine-tuned models.

Ax Yannis Montreuil, Letian Yu, Axel Carlier, Lai Xing Ng, Wei Tsang Ooi 14h ago

Adversarial Robustness in One-Stage Learning-to-Defer

Analyzes adversarial robustness in learning-to-defer systems where inputs are routed to predictors or experts, extending prior two-stage analyses to one-stage joint training.

Ax Sangwon Jang, Taekyung Ki, Jaehyeong Jo, Saining Xie, Jaehong Yoon, Sung Ju Hwang 14h ago

Self-Refining Video Sampling

Self-refining video sampling method improving physical realism in generated videos without external verifiers or augmented training.