Ax Deepak Pandita, Flip Korn, Chris Welty, Christopher M. Homan 5/14/2026

Improving Reproducibility in Evaluation through Multi-Level Annotator Modeling

Research on improving LLM evaluation reproducibility by modeling annotator biases in human rating systems for AI safety assessment.

Ax Bethel Hall, William Eiers 5/14/2026

Neurosymbolic Auditing of Natural-Language Software Requirements

Neurosymbolic approach combining LLMs with SMT solvers to audit natural-language software requirements for ambiguity and inconsistency.

Ax Harry Mayne, Lev McKinney, Jan Dubi\'nski, Adam Karvonen, James Chua, Owain Evans 5/14/2026

Negation Neglect: When models fail to learn negations in training

Negation Neglect: LLMs fail to learn negations during finetuning, believing falsified claims despite recognizing them as false in context.

Ax Tara Bogavelli, Gabrielle Gauthier Melan\c{c}on, Katrina Stankiewicz, Oluwanifemi Bamgbose, Fanny Riols, Hoang H. Nguyen, Raghav Mehndiratta, Lindsay Devon Brin, Joseph Marinier, Hari Subramani, Anil Madamala, Sridhar Krishna Nemala, Srinivas Sunkara 5/14/2026

EVA-Bench: A New End-to-end Framework for Evaluating Voice Agents

EVA-Bench: Evaluation framework for voice agents addressing realistic conversation simulation and voice-specific failure mode measurement.

Ax Ziheng Zhang, Yunzhong Hou, Naijing Liu, Liang Zheng 5/14/2026

WARDEN: Endangered Indigenous Language Transcription and Translation with 6 Hours of Training Data

WARDEN: Language model for transcribing and translating endangered Wardaman language using only 6 hours of annotated audio data.

Ax Yanjie Li, Lina Yu, Weijun Li, Min Wu, Liping Zhang, Jingyi Liu, Yusong Deng, Mingzhu Wan, Xin Ning 5/14/2026

ChatSR: Multimodal Large Language Models for Scientific Formula Discovery

ChatSR: Multimodal LLM for scientific formula discovery from graphs and images with specialized scientific data understanding.

Ax Chen Chen, Xueluan Gong, Ziyao Liu, Weifeng Jiang, Si Qi Goh, Kwok-Yan Lam 5/14/2026

AI Safety Landscape for Large Language Models: Taxonomy, State-of-the-art, and Future Directions

Taxonomy and survey of AI safety landscape for large language models covering design, development, adoption and deployment impacts.

Ax Shiguang Wu, Yaqing Wang, Quanming Yao 5/14/2026

Language Model Networks: Supervision-Efficient Learning through Dense Communication

Language model networks study: pre-trained LMs as reusable nodes in inference systems with learned communication patterns instead of natural language.

Ax Kangye Ji, Yuan Meng, Hanyun Cui, Ye Li, Jianbo Zhou, Shengjia Hua, Lei Chen, Zhi Wang 5/14/2026

Block-wise Adaptive Caching for Accelerating Diffusion Policy

Block-wise adaptive caching technique to accelerate Diffusion Policy inference for real-time robotic control.

Ax Fengxiao Fan, Jingzhe Ni, Xiaolong Yin, Sirui Wang, Xingyu Lu, Qiang Zou, Ruofeng Tong, Min Tang, Peng Du 5/14/2026

CADDesigner: Conceptual CAD Model Generation with a General-Purpose Agent

CADDesigner: LLM-powered agent for conceptual CAD modeling accepting text and sketches with interactive refinement dialogue.

Ax Tianyi Ma, Yue Zhang, Zehao Wang, Parisa Kordjamshidi 5/14/2026

Breaking Down and Building Up: Mixture of Skill-Based Vision-and-Language Navigation Agents

SkillNav: Modular framework for vision-language navigation using skill-based decomposition to improve generalization on complex spatial-temporal reasoning.

Ax Terry Jingchen Zhang, Gopal Dev, Ning Wang, Max Obreiter, Punya Syon Pandey, Keenan Samway, Wenyuan Jiang, Yinya Huang, Bernhard Sch\"olkopf, Mrinmaya Sachan, Zhijing Jin 5/14/2026

Test of Time: Rethinking Temporal Signal of Benchmark Contamination

Critical analysis of temporal signals in benchmark contamination detection, showing sensitivity to question construction independent of data memorization.

Ax Jordan Abi Nader, David Lee, Nathaniel Dennler, Andreea Bobu 5/14/2026

QuickLAP: Quick Language-Action Preference Learning for Semi-Autonomous Systems

QuickLAP: Bayesian framework for robot learning that fuses physical corrections and language feedback to infer reward functions.

Ax Mingwei Li, Xiaoyuan Zhang, Chengwei Yang, Zilong Zheng, Yaodong Yang 5/14/2026

Prismatic World Model: Learning Compositional Dynamics for Planning in Hybrid Systems

Prismatic world model for robotic planning that learns compositional dynamics separately for distinct physical modes like contact/impact events.

Ax Youngmin Im, Byeongung Jo, Jaeyoung Wi, Seungwoo Baek, Tae Hoon Min, Joo Hyung Lee, Sangeun Oh, Insik Shin, Sunjae Lee 5/14/2026

MobiBench: Multi-Branch, Modular Benchmark for Mobile GUI Agents

MobiBench: Multimodal benchmark for mobile GUI agents addressing limitations of existing offline/online benchmarks with multiple valid action paths.

Ax Sitao Cheng, Tianle Li, Xuhan Huang, Xunjian Yin, Difan Zou 5/14/2026

Differentiable Evolutionary Reinforcement Learning

Differentiable Evolutionary RL optimizes reward functions using gradient information to improve policy performance on complex reasoning tasks.

Ax Yibo Lyu, Gongwei Chen, Rui Shao, Weili Guan, Liqiang Nie 5/14/2026

PersonalAlign: Hierarchical Implicit Intent Alignment for Personalized GUI Agent with Long-Term User-Centric Records

PersonalAlign: GUI agent framework that aligns with implicit user intents by leveraging long-term user records as persistent context for personalized task completion.

Ax Fabian Fumagalli, R. Teal Witter, Christopher Musco 5/14/2026

PolySHAP: Extending KernelSHAP with Interaction-Informed Polynomial Regression

PolySHAP improves KernelSHAP approximation of Shapley values for explainable AI by using polynomial regression instead of linear approximation to reduce computational cost.

Ax Pranav Mahajan, Ihor Kendiukhov, Syed Hussain, Lydia Nottingham 5/14/2026

Mind the Gap: How Elicitation Protocols Shape the Stated-Revealed Preference Gap in Language Models

Study examining how elicitation protocol design affects stated-revealed preference gaps across 24 language models.

Ax Seanie Lee, Sangwoo Park, Yumin Choi, Gyeongman Kim, Minki Kang, Jihun Yun, Dongmin Park, Jongho Park, Sung Ju Hwang 5/14/2026

THINKSAFE: Self-Generated Safety Alignment for Reasoning Models

THINKSAFE: Safety alignment approach for reasoning models that self-generates safety constraints without external teacher distillation.

Ax Xingyuan Hua, Sheng Yue, Xinyi Li, Yizhe Zhao, Jinrui Zhang, Ju Ren 5/14/2026

Context Learning for Multi-Agent Discussion

M2CL: Multi-LLM context learning method for multi-agent discussion systems addressing discussion inconsistency and context misalignment.

Ax Tae Soo Kim, Yoonjoo Lee, Jaesang Yu, John Joon Young Chung, Juho Kim 5/14/2026

DiscoverLLM: From Executing Intents to Discovering Them

DiscoverLLM: Method enabling LLMs to help users discover intents through interactive exploration rather than just executing stated requests.

Ax Shengyue Guan, Yihao Liu, Lang Cao 5/14/2026

SupChain-Bench: Benchmarking Large Language Models for Real-World Supply Chain Management

SupChain-Bench: Benchmark for evaluating LLMs on real-world supply chain management with multi-step domain-specific orchestration.

Ax Haoran Zheng, Chen Shani 5/14/2026

When to Think Fast and Slow? AMOR: Adaptive Entropy Gate for Hybrid Models

AMOR: Hybrid architecture combining recurrence and attention, selectively invoking attention based on predictive uncertainty.

Ax Yun-Shiuan Chuang, Chaitanya Kulkarni, Alec Chiu, Avinash Thangali, Zijie Pan, Shivani Shekhar, Yirou Ge, Yixi Li, Uma Kona, Linsey Pang, Prakhar Mehrotra 5/14/2026

Toward Scalable Verifiable Reward: Proxy State-Based Evaluation for Multi-turn Tool-Calling LLM Agents

Proxy State-Based Evaluation: LLM-driven benchmark method for multi-turn tool-calling agents that scales beyond deterministic backends.

Ax Junkai Lu, Peng Chen, Xingjian Wu, Yang Shu, Chenjuan Guo, Christian S. Jensen, Bin Yang 5/14/2026

PATRA: Pattern-Aware Alignment and Balanced Reasoning for Time Series Question Answering

LLM-based approach for time series question answering using pattern alignment and balanced reasoning across task complexity.

Ax Baoqing Yue, Zihan Zhu, Yutong Han, Qian Sun, Jichen Feng, Hufei Yang, Yifan Zhang, Mengdi Wang 5/14/2026

Interactive Benchmarks

Interactive Benchmarks: Evaluation paradigm assessing model reasoning by testing ability to decide what information to acquire and use.

Ax Swarna Kamal Paul, Shubhendu Sharma, Nitin Sareen 5/14/2026

GAAMA: Graph Augmented Associative Memory for Agents

GAAMA: Graph-augmented memory architecture for AI agents to maintain coherent long-term personalized behavior across sessions.

Ax Yu Li, Haoyu Luo, Yuejin Xie, Yuqian Fu, Zhonghao Yang, Shuai Shao, Qihan Ren, Wanying Qu, Yanwei Fu, Yujiu Yang, Jing Shao, Xia Hu, Dongrui Liu 5/14/2026

ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis

Trajectory-level safety benchmark (ATBench) for evaluating LLM-based agents across realistic multi-step interactions with diverse failure modes.

Ax Wanyi Chen, Xiao Yang, Xu Yang, Tianming Sha, Qizheng Li, Zhuo Wang, Bowen Xian, Fang Kong, Weiqing Liu, Jiang Bian 5/14/2026

Agent^2 RL-Bench: Can LLM Agents Engineer Agentic RL Post-Training?

Benchmark evaluating whether LLM agents can autonomously design, implement, and execute RL post-training pipelines for model improvement.

Ax Charles Koutcheme, Juho Leinonen, Arto Hellas 5/14/2026

Teaching Language Models How to Code Like Learners: Conversational Serialization for Student Simulation

Framework for training open-weight language models to simulate student coding behavior for educational tutoring system evaluation.

Ax Philip Zhong, Don Wang, Jason Zhang 5/14/2026

Evaluating AI Meeting Summaries with a Reusable Cross-Domain Pipeline

Industrial evaluation system for LLM-generated meeting summaries with structured ground-truth construction and privacy-bounded monitoring.

Ax Jackson Vonderhorst, Kuangshi Ai, Haichao Miao, Shusen Liu, Chaoli Wang 5/14/2026

Exploring Interaction Paradigms for LLM Agents in Scientific Visualization

Evaluates three LLM agent interaction paradigms (structured tools, computer-use, coding agents) on scientific visualization tasks across 15 benchmarks.

Ax Guowei Zou, Haitao Wang, Beiwen Zhang, Boning Zhang, Hejun Wu 5/14/2026

CoFlow: Coordinated Few-Step Flow for Offline Multi-Agent Decision Making

Study on coordinated flow models for offline multi-agent reinforcement learning that balances efficiency and coordination.

Ax Aleksandr Bowkis, Marie Davidsen Buhl, Jacob Pfau, Geoffrey Irving 5/14/2026

Automated alignment is harder than you think

Paper arguing automated alignment via research agents risks producing misleading safety assessments without deliberate sabotage.

Ax Daniel Zheng, Ingrid von Glehn, Yori Zwols, Iuliya Beloshapka, Lars Buesing, Daniel M. Roy, Martin Wattenberg, Bogdan Georgiev, Tatiana Schmidt, Andrew Cowie, Fernanda Viegas, Dimitri Kanevsky, Vineet Kahlon, Hartmut Maennel, Sophia Alj, George Holland, Alex Davies, Pushmeet Kohli 5/14/2026

AI co-mathematician: Accelerating mathematicians with agentic AI

AI co-mathematician workbench enabling mathematicians to collaboratively leverage AI agents for research including ideation, computation, and theorem proving.

Ax Sixing Chen, Ji-An Li, Saner Cakir, Sinan Akcali, Kayla Lee, Marcelo G. Mattar 5/14/2026

Extracting Search Trees from LLM Reasoning Traces Reveals Myopic Planning

Method to extract and analyze search trees from LLM reasoning traces to quantify planning capabilities and identify myopic decision-making.

Ax Roger Creus Castanyer, Pablo Samuel Castro, Glen Berseth 5/14/2026

Agentick: A Unified Benchmark for General Sequential Decision-Making Agents

Agentick: Unified benchmark enabling fair comparison of RL, LLM, VLM, and hybrid agents on sequential decision-making tasks.

Ax Jackson Clark, Yiming Su, Saad Mohammad Rafid Pial, Yifang Tian, Lily Gniedziejko, Hans-Arno Jacobsen, Yinfang Chen, Tianyin Xu 5/14/2026

SREGym: A Live Benchmark for AI SRE Agents with High-Fidelity Failure Scenarios

SREGym: High-fidelity benchmark for AI SRE agents with live cloud-native systems and realistic failure scenarios for diagnosis and mitigation.

Ax Yuanhao Li, Hongbo Wang, Xiaotang Shang, Xunzhu Tang, Yiming Cao, Xuhong Chen 5/14/2026

BoostAPR: Boosting Automated Program Repair via Execution-Grounded Reinforcement Learning with Dual Reward Models

BoostAPR: Three-stage RL framework for automated program repair using execution feedback and dual reward models for credit assignment.

Ax Shawn Li, Chenxiao Yu, Han Wang, Wei Yang, Ryan Rossi, Franck Dernoncourt, Xiyang Hu, Philip Yu, Chaowei Xiao, Huan Zhang, Yue Zhao 5/14/2026

FORTIS: Benchmarking Over-Privilege in Agent Skills

FORTIS benchmark evaluating privilege escalation vulnerabilities in LLM agent skill layers and access control policies.

Ax Haoqiang Kang, Xiaokang Ye, Yuhan Liu, Siddhant Hitesh Mantri, Lingjun Mao, James Fleming, Drishti Regmi, Lianhui Qin 5/14/2026

SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learning

SimWorld Studio: System using evolving coding agents to automatically generate diverse 3D environments for embodied agent training.

Ax Yuyang Dai, Zheng Chen, Jathurshan Pradeepkumar, Yasuko Matsubara, Jimeng Sun, Yasushi Sakurai, Yushun Dong 5/14/2026

EpiGraph: Building Generalists for Evidence-Intensive Epilepsy Reasoning in the Wild

EpiGraph knowledge graph and benchmark for evaluating knowledge-augmented clinical reasoning in epilepsy diagnosis from heterogeneous evidence.

Ax Mingxiong Lin, Zhangquan Gong, Maowen Tang, Qian Li, Chuangchuang Wang, Jian Ma, Sutian Huang, Kai Tang, Haonan Lu 5/14/2026

expo: Exploration-prioritized policy optimization via adaptive kl regulation and gaussian curriculum sampling

Expo: Improved RL policy optimization for LLM reasoning with adaptive KL regulation and curriculum sampling over GRPO baseline.

Ax Songlin Bai, Xintong Wang, Linlin Yu, Bin Chen, Zhiang Xu, Yuyang Sheng, Changtong Zan, Xiaofeng Zhu, Yizhe Zhang, Jiru Li, Mingze Guo, Ling Zou, Yalong Li, Chengfu Huo, Liang Ding 5/14/2026

IndustryBench: Probing the Industrial Knowledge Boundaries of LLMs

IndustryBench: 2,049-item benchmark testing LLM knowledge on industrial procurement with safety-critical constraints and standards compliance.

Ax Yanjie Li, Liping Zhang, Min Wu, Weijun Li, Lina Yu, Jingyi Liu, Yusong Deng, Mingzhu Wan, Xin Ning 5/14/2026

GESR: A Genetic Programming-Based Symbolic Regression Method with Gene Editing

Genetic programming approach to symbolic regression with gene editing for discovering mathematical formulas from scientific data.

Ax Duc Anh Le, Tien-Phat Nguyen, Thien Huu Nguyen, Linh Ngo Van, Trung Le 5/14/2026

Selective Off-Policy Reference Tuning with Plan Guidance

SORT method for reinforcement learning that adds repair updates for failed rollouts using plan guidance and token probability weighting.

Ax ShiYing Huang, Liang Lin, Yuer Li, Kaiwen Luo, Zhenhong Zhou, An Zhang, Junhao Dong, Kun Wang, Zhigang Zeng 5/14/2026

Explaining and Breaking the Safety-Helpfulness Ceiling via Preference Dimensional Expansion

Study on resolving safety-helpfulness trade-offs in LLM alignment through preference dimensional expansion approach.

Ax Willow Mandil, Amir Ghalamzan-E 5/14/2026

Multi-Modal World Model for Physical Robot Interactions: Simultaneous Visual and Tactile Predictions for Enhanced Accuracy

Research on multi-modal world models integrating tactile and visual feedback for predicting robotic action outcomes in complex environments.

Ax Peter Baile Chen, Devin Yang, Weiyue Li, Fabian Wenz, Yi Zhang, Nesime Tatbul, Michael Cafarella, \c{C}a\u{g}atay Demiralp, Michael Stonebraker 5/14/2026

BEAVER: An Enterprise Benchmark for Text-to-SQL

Benchmark for evaluating LLM text-to-SQL performance on complex enterprise databases with intricate schemas and domain knowledge requirements.