Tootfinder

How Reinforcement Learning After Next-Token Prediction Facilitates Learning
Nikolaos Tsilivis, Eran Malach, Karen Ullrich, Julia Kempe
https://arxiv.org/abs/2510.11495 https://

How Reinforcement Learning After Next-Token Prediction Facilitates Learning
Recent advances in reasoning domains with neural networks have primarily been enabled by a training recipe that optimizes Large Language Models, previously trained to predict the next-token in a sequence, with reinforcement learning algorithms. We introduce a framework to study the success of this paradigm, and we theoretically expose the optimization mechanisms by which reinforcement learning improves over next-token prediction in this setting. We study learning from mixture distributions of s…

CoIRL-AD: Collaborative-Competitive Imitation-Reinforcement Learning in Latent World Models for Autonomous Driving
Xiaoji Zheng, Ziyuan Yang, Yanhao Chen, Yuhang Peng, Yuanrong Tang, Gengyuan Liu, Bokui Chen, Jiangtao Gong
https://arxiv.org/abs/2510.12560

CoIRL-AD: Collaborative-Competitive Imitation-Reinforcement Learning in Latent World Models for Autonomous Driving
End-to-end autonomous driving models trained solely with imitation learning (IL) often suffer from poor generalization. In contrast, reinforcement learning (RL) promotes exploration through reward maximization but faces challenges such as sample inefficiency and unstable convergence. A natural solution is to combine IL and RL. Moving beyond the conventional two-stage paradigm (IL pretraining followed by RL fine-tuning), we propose CoIRL-AD, a competitive dual-policy framework that enables IL an…

Residual MPC: Blending Reinforcement Learning with GPU-Parallelized Model Predictive Control
Se Hwan Jeon, Ho Jae Lee, Seungwoo Hong, Sangbae Kim
https://arxiv.org/abs/2510.12717

Residual MPC: Blending Reinforcement Learning with GPU-Parallelized Model Predictive Control
Model Predictive Control (MPC) provides interpretable, tunable locomotion controllers grounded in physical models, but its robustness depends on frequent replanning and is limited by model mismatch and real-time computational constraints. Reinforcement Learning (RL), by contrast, can produce highly robust behaviors through stochastic training but often lacks interpretability, suffers from out-of-distribution failures, and requires intensive reward engineering. This work presents a GPU-paralleli…

On Sunday, October 5, 2025, my wife and I are participating in the Canadian Cancer Society's Run for the Cure. This national fundraiser brings together communities across the country who will run or walk in support of all Canadians impacted by breast cancer. We are walking in honour of my mother, my two sisters, and both our daughters - all of whom have been touched by this terrible disease.
Our family isn't special as 1 in 8 women will face a breast cancer diagnosis in their lifetime, It is the most commonly diagnosed cancer among Canadian women, and all of us likely know someone who has battled it in the past, or is fighting it today. I encourage all Canadians to donate in honour of someone they know or simply to help their communities fight breast cancer. This cause is particularly important to us, especially this year, as we support our daughters, and I hope all Canadians get involved.
#CanadianCancerSociety #RunForTheCure
https://support.cancer.ca/site/TR/RunfortheCure/RFTC_NW_even_?px=15178695&pg=personal&fr_id=30455

Compositional shield synthesis for safe reinforcement learning in partial observability
Steven Carr, Georgios Bakirtzis, Ufuk Topcu
https://arxiv.org/abs/2509.12085 https://

Compositional shield synthesis for safe reinforcement learning in partial observability
Agents controlled by the output of reinforcement learning (RL) algorithms often transition to unsafe states, particularly in uncertain and partially observable environments. Partially observable Markov decision processes (POMDPs) provide a natural setting for studying such scenarios with limited sensing. Shields filter undesirable actions to ensure safe RL by preserving safety requirements in the agents' policy. However, synthesizing holistic shields is computationally expensive in complex depl…

Strategic Cyber Defense via Reinforcement Learning-Guided Combinatorial Auctions
Mai Pham, Vikrant Vaze, Peter Chin
https://arxiv.org/abs/2509.10983 https://

Strategic Cyber Defense via Reinforcement Learning-Guided Combinatorial Auctions
Cyber defense operations increasingly require long-term strategic planning under uncertainty and resource constraints. We propose a new use of combinatorial auctions for allocating defensive action bundles in a realistic cyber environment, using host-specific valuations derived from reinforcement learning (RL) Q-values. These Q-values encode long-term expected utility, allowing upstream planning. We train CAFormer, a differentiable Transformer-based auction mechanism, to produce allocations tha…

Biased-Attention Guided Risk Prediction for Safe Decision-Making at Unsignalized Intersections
Chengyang Dong, Nan Guo
https://arxiv.org/abs/2510.12428 https://

Biased-Attention Guided Risk Prediction for Safe Decision-Making at Unsignalized Intersections
Autonomous driving decision-making at unsignalized intersections is highly challenging due to complex dynamic interactions and high conflict risks. To achieve proactive safety control, this paper proposes a deep reinforcement learning (DRL) decision-making framework integrated with a biased attention mechanism. The framework is built upon the Soft Actor-Critic (SAC) algorithm. Its core innovation lies in the use of biased attention to construct a traffic risk predictor. This predictor assesses …

Thought Purity: Defense Paradigm For Chain-of-Thought Attack
Zihao Xue, Zhen Bi, Long Ma, Zhenlin Hu, Yan Wang, Zhenfang Liu, Qing Sheng, Jie Xiao, Jungang Lou
https://arxiv.org/abs/2507.12314

Thought Purity: Defense Paradigm For Chain-of-Thought Attack
While reinforcement learning-trained Large Reasoning Models (LRMs, e.g., Deepseek-R1) demonstrate advanced reasoning capabilities in the evolving Large Language Models (LLMs) domain, their susceptibility to security threats remains a critical vulnerability. This weakness is particularly evident in Chain-of-Thought (CoT) generation processes, where adversarial methods like backdoor prompt attacks can systematically subvert the model's core reasoning mechanisms. The emerging Chain-of-Thought Atta…

Real-Time Defense Against Coordinated Cyber-Physical Attacks: A Robust Constrained Reinforcement Learning Approach
Saman Mazaheri Khamaneh, Tong Wu, Wei Sun, Cong Chen
https://arxiv.org/abs/2509.10999 …

Real-Time Defense Against Coordinated Cyber-Physical Attacks: A Robust Constrained Reinforcement Learning Approach
Modern power systems face increasing vulnerability to sophisticated cyber-physical attacks beyond traditional N-1 contingency frameworks. Existing security paradigms face a critical bottleneck: efficiently identifying worst-case scenarios and rapidly coordinating defensive responses are hindered by intensive computation and time delays, during which cascading failures can propagate. This paper presents a novel tri-level robust constrained reinforcement learning (RCRL) framework for robust power…

Few-shot Vision-based Human Activity Recognition with MLLM-based Visual Reinforcement Learning
Wenqi Zheng, Yutaka Arakawa
https://arxiv.org/abs/2508.10371 https://

Few-shot Vision-based Human Activity Recognition with MLLM-based Visual Reinforcement Learning
Reinforcement learning in large reasoning models enables learning from feedback on their outputs, making it particularly valuable in scenarios where fine-tuning data is limited. However, its application in multi-modal human activity recognition (HAR) domains remains largely underexplored. Our work extends reinforcement learning to the human activity recognition domain with multimodal large language models. By incorporating visual reinforcement learning in the training process, the model's gener…