Tootfinder

LESER: Learning to Expand via Search Engine-feedback Reinforcement in e-Commerce
Yipeng Zhang, Bowen Liu, Xiaoshuang Zhang, Aritra Mandal, Zhe Wu, Canran Xu
https://arxiv.org/abs/2509.05570

LESER: Learning to Expand via Search Engine-feedback Reinforcement in e-Commerce
User queries in e-commerce search are often vague, short, and underspecified, making it difficult for retrieval systems to match them accurately against structured product catalogs. This challenge is amplified by the one-to-many nature of user intent, where a single query can imply diverse and competing needs. Existing methods, including neural query expansion and prompting-based LLM approaches, fall short in real-world settings: they struggle to capture nuanced user intent, often generate outp…

Learning to Capture Rocks using an Excavator: A Reinforcement Learning Approach with Guiding Reward Formulation
Amirmasoud Molaei, Reza Ghabcheloo
https://arxiv.org/abs/2510.04168

Learning to Capture Rocks using an Excavator: A Reinforcement Learning Approach with Guiding Reward Formulation
Rock capturing with standard excavator buckets is a challenging task typically requiring the expertise of skilled operators. Unlike soil digging, it involves manipulating large, irregular rocks in unstructured environments where complex contact interactions with granular material make model-based control impractical. Existing autonomous excavation methods focus mainly on continuous media or rely on specialized grippers, limiting their applicability to real-world construction sites. This paper i…

AQA-TTRL: Self-Adaptation in Audio Question Answering with Test-Time Reinforcement Learning
Haoyu Zhang, Jiaxian Guo, Yusuke Iwasawa, Yutaka Matsuo
https://arxiv.org/abs/2510.05478

AQA-TTRL: Self-Adaptation in Audio Question Answering with Test-Time Reinforcement Learning
Large Audio Language Models (LALMs) demonstrate impressive general audio understanding, but once deployed, they are static and fail to improve with new real-world audio data. As traditional supervised fine-tuning is costly, we introduce a novel framework for test-time audio understanding, AQA-TTRL, where an LALM evolves on-the-fly using only unlabeled test data. It first generates pseudo-labels from the prediction via majority voting, then optimizes the model via reinforcement learning. To hand…

Reinforcement Learning with Action-Triggered Observations
Alexander Ryabchenko, Wenlong Mou
https://arxiv.org/abs/2510.02149 https://arxiv.org/pdf/2510.021…

Reinforcement Learning with Action-Triggered Observations
We study reinforcement learning problems where state observations are stochastically triggered by actions, a constraint common in many real-world applications. This framework is formulated as Action-Triggered Sporadically Traceable Markov Decision Processes (ATST-MDPs), where each action has a specified probability of triggering a state observation. We derive tailored Bellman optimality equations for this framework and introduce the action-sequence learning paradigm in which agents commit to ex…

#HotTake:
Liberalism is a syncretic religion where the highest deity is the State and all other deities are subject to it. Liberalism asserts this distinction by classifying political systems as deriving from "natural law" and religions as faith systems. Classical liberalism divided the world into "primitive" beliefs and "enlightened" beliefs. Atheistic anarchism aligns with this distinction by opposing the state and capitalism through logic, rather than asserting that the state and capitalism are themselves systems of faith. Atheistic anarchism further reinforces the narrative of the liberal state by aligning with the liberal understanding of religion as a "primitive" institution.
Graeber et al pointed out the connections between property rights as a form of fedishism (magic) and the evolution of currency, debt, taxes, and the state from temples.

Mutual Information Tracks Policy Coherence in Reinforcement Learning
Cameron Reid, Wael Hafez, Amirhossein Nazeri
https://arxiv.org/abs/2509.10423 https://…

Mutual Information Tracks Policy Coherence in Reinforcement Learning
Reinforcement Learning (RL) agents deployed in real-world environments face degradation from sensor faults, actuator wear, and environmental shifts, yet lack intrinsic mechanisms to detect and diagnose these failures. We present an information-theoretic framework that reveals both the fundamental dynamics of RL and provides practical methods for diagnosing deployment-time anomalies. Through analysis of state-action mutual information patterns in a robotic control task, we first demonstrate that…

Mining the Long Tail: A Comparative Study of Data-Centric Criticality Metrics for Robust Offline Reinforcement Learning in Autonomous Motion Planning
Antonio Guillen-Perez
https://arxiv.org/abs/2508.18397

Mining the Long Tail: A Comparative Study of Data-Centric Criticality Metrics for Robust Offline Reinforcement Learning in Autonomous Motion Planning
Offline Reinforcement Learning (RL) presents a promising paradigm for training autonomous vehicle (AV) planning policies from large-scale, real-world driving logs. However, the extreme data imbalance in these logs, where mundane scenarios vastly outnumber rare "long-tail" events, leads to brittle and unsafe policies when using standard uniform data sampling. In this work, we address this challenge through a systematic, large-scale comparative study of data curation strategies designed to focus …

Learning Real-World Acrobatic Flight from Human Preferences
Colin Merk, Ismail Geles, Jiaxu Xing, Angel Romero, Giorgia Ramponi, Davide Scaramuzza
https://arxiv.org/abs/2508.18817

Learning Real-World Acrobatic Flight from Human Preferences
Preference-based reinforcement learning (PbRL) enables agents to learn control policies without requiring manually designed reward functions, making it well-suited for tasks where objectives are difficult to formalize or inherently subjective. Acrobatic flight poses a particularly challenging problem due to its complex dynamics, rapid movements, and the importance of precise execution. In this work, we explore the use of PbRL for agile drone control, focusing on the execution of dynamic maneuve…

Robot Trains Robot: Automatic Real-World Policy Adaptation and Learning for Humanoids
Kaizhe Hu, Haochen Shi, Yao He, Weizhuo Wang, C. Karen Liu, Shuran Song
https://arxiv.org/abs/2508.12252

Robot Trains Robot: Automatic Real-World Policy Adaptation and Learning for Humanoids
Simulation-based reinforcement learning (RL) has significantly advanced humanoid locomotion tasks, yet direct real-world RL from scratch or adapting from pretrained policies remains rare, limiting the full potential of humanoid robots. Real-world learning, despite being crucial for overcoming the sim-to-real gap, faces substantial challenges related to safety, reward design, and learning efficiency. To address these limitations, we propose Robot-Trains-Robot (RTR), a novel framework where a rob…

Sight Over Site: Perception-Aware Reinforcement Learning for Efficient Robotic Inspection
Richard Kuhlmann, Jakob Wolfram, Boyang Sun, Jiaxu Xing, Davide Scaramuzza, Marc Pollefeys, Cesar Cadena
https://arxiv.org/abs/2509.17877

Sight Over Site: Perception-Aware Reinforcement Learning for Efficient Robotic Inspection
Autonomous inspection is a central problem in robotics, with applications ranging from industrial monitoring to search-and-rescue. Traditionally, inspection has often been reduced to navigation tasks, where the objective is to reach a predefined location while avoiding obstacles. However, this formulation captures only part of the real inspection problem. In real-world environments, the inspection targets may become visible well before their exact coordinates are reached, making further movemen…