Tootfinder

Learning Interpretable Rules from Neural Networks: Neurosymbolic AI for Radar Hand Gesture Recognition
Sarah Seifi, Tobias Sukianto, Cecilia Carbonelli, Lorenzo Servadei, Robert Wille
https://arxiv.org/abs/2506.22443

Learning Interpretable Rules from Neural Networks: Neurosymbolic AI for Radar Hand Gesture Recognition
Rule-based models offer interpretability but struggle with complex data, while deep neural networks excel in performance yet lack transparency. This work investigates a neuro-symbolic rule learning neural network named RL-Net that learns interpretable rule lists through neural optimization, applied for the first time to radar-based hand gesture recognition (HGR). We benchmark RL-Net against a fully transparent rule-based system (MIRA) and an explainable black-box model (XentricAI), evaluating a…

DiG-Net: Enhancing Quality of Life through Hyper-Range Dynamic Gesture Recognition in Assistive Robotics
Eran Bamani Beeri, Eden Nissinman, Avishai Sintov
https://arxiv.org/abs/2505.24786

DiG-Net: Enhancing Quality of Life through Hyper-Range Dynamic Gesture Recognition in Assistive Robotics
Dynamic hand gestures play a pivotal role in assistive human-robot interaction (HRI), facilitating intuitive, non-verbal communication, particularly for individuals with mobility constraints or those operating robots remotely. Current gesture recognition methods are mostly limited to short-range interactions, reducing their utility in scenarios demanding robust assistive communication from afar. In this paper, we introduce a novel approach designed specifically for assistive robotics, enabling …

Coordinated 2D-3D Visualization of Volumetric Medical Data in XR with Multimodal Interactions
Qixuan Liu, Shi Qiu, Yinqiao Wang, Xiwen Wu, Kenneth Siu Ho Chok, Chi-Wing Fu, Pheng-Ann Heng
https://arxiv.org/abs/2506.22926

Coordinated 2D-3D Visualization of Volumetric Medical Data in XR with Multimodal Interactions
Volumetric medical imaging technologies produce detailed 3D representations of anatomical structures. However, effective medical data visualization and exploration pose significant challenges, especially for individuals with limited medical expertise. We introduce a novel XR-based system with two key innovations: (1) a coordinated visualization module integrating Multi-layered Multi-planar Reconstruction with 3D mesh models and (2) a multimodal interaction framework combining hand gestures with…

Diver-Robot Communication Dataset for Underwater Hand Gesture Recognition
Igor Kvasi\'c, Derek Orbaugh Antillon, {\DH}ula Na{\dj}, Christopher Walker, Iain Anderson, Nikola Mi\v{s}kovi\'c
https://arxiv.org/abs/2506.08974

Diver-Robot Communication Dataset for Underwater Hand Gesture Recognition
In this paper, we present a dataset of diving gesture images used for human-robot interaction underwater. By offering this open access dataset, the paper aims at investigating the potential of using visual detection of diving gestures from an autonomous underwater vehicle (AUV) as a form of communication with a human diver. In addition to the image recording, the same dataset was recorded using a smart gesture recognition glove. The glove uses elastomer sensors and on-board processing to determ…

Multimodal Emotion Coupling via Speech-to-Facial and Bodily Gestures in Dyadic Interaction
Von Ralph Dane Marquez Herbuela, Yukie Nagai
https://arxiv.org/abs/2506.10010

Multimodal Emotion Coupling via Speech-to-Facial and Bodily Gestures in Dyadic Interaction
Human emotional expression emerges through coordinated vocal, facial, and gestural signals. While speech face alignment is well established, the broader dynamics linking emotionally expressive speech to regional facial and hand motion remains critical for gaining a deeper insight into how emotional and behavior cues are communicated in real interactions. Further modulating the coordination is the structure of conversational exchange like sequential turn taking, which creates stable temporal win…

Multimodal "Puppeteer": An Exploration of Robot Teleoperation Via Virtual Counterpart with LLM-Driven Voice and Gesture Interaction in Augmented Reality
Yuchong Zhang, Bastian Orthmann, Shichen Ji, Michael Welle, Jonne Van Haastregt, Danica Kragic
https://arxiv.org/abs/2506.13189…

Multimodal "Puppeteer": An Exploration of Robot Teleoperation Via Virtual Counterpart with LLM-Driven Voice and Gesture Interaction in Augmented Reality
The integration of robotics and augmented reality (AR) holds transformative potential for advancing human-robot interaction (HRI), offering enhancements in usability, intuitiveness, accessibility, and collaborative task performance. This paper introduces and evaluates a novel multimodal AR-based robot puppeteer framework that enables intuitive teleoperation via virtual counterpart through large language model (LLM)-driven voice commands and hand gesture interactions. Utilizing the Meta Quest 3,…

Real-Time Confidence Detection through Facial Expressions and Hand Gestures
Tanjil Hasan Sakib, Samia Jahan Mojumder, Rajan Das Gupta, Md Imrul Hasan Showmick, Md. Yeasin Rahat, Md. Jakir Hossen
https://arxiv.org/abs/2506.09153

Real-Time Confidence Detection through Facial Expressions and Hand Gestures
Real-time face orientation recognition is a cutting-edge technology meant to track and analyze facial movements in virtual environments such as online interviews, remote meetings, and virtual classrooms. As the demand for virtual interactions grows, it becomes increasingly important to measure participant engagement, attention, and overall interaction. This research presents a novel solution that leverages the Media Pipe Face Mesh framework to identify facial landmarks and extract geometric dat…