Tootfinder

Vision-Based Assistive Technologies for People with Cerebral Visual Impairment: A Review and Focus Study
Bhanuka Gamage, Leona Holloway, Nicola McDowell, Thanh-Toan Do, Nicholas Price, Arthur Lowery, Kim Marriott
https://arxiv.org/abs/2505.22983

Vision-Based Assistive Technologies for People with Cerebral Visual Impairment: A Review and Focus Study
Over the past decade, considerable research has investigated Vision-Based Assistive Technologies (VBAT) to support people with vision impairments to understand and interact with their immediate environment using machine learning, computer vision, image enhancement, and/or augmented/virtual reality. However, this has almost totally overlooked a growing demographic: people with Cerebral Visual Impairment (CVI). Unlike ocular vision impairments, CVI arises from damage to the brain's visual process…

Disrupting Vision-Language Model-Driven Navigation Services via Adversarial Object Fusion
Chunlong Xie, Jialing He, Shangwei Guo, Jiacheng Wang, Shudong Zhang, Tianwei Zhang, Tao Xiang
https://arxiv.org/abs/2505.23266

Disrupting Vision-Language Model-Driven Navigation Services via Adversarial Object Fusion
We present Adversarial Object Fusion (AdvOF), a novel attack framework targeting vision-and-language navigation (VLN) agents in service-oriented environments by generating adversarial 3D objects. While foundational models like Large Language Models (LLMs) and Vision Language Models (VLMs) have enhanced service-oriented navigation systems through improved perception and decision-making, their integration introduces vulnerabilities in mission-critical service workflows. Existing adversarial attac…

This https://arxiv.org/abs/2412.19297 has been replaced.
initial toot: https://mastoxiv.page/@…

The XY model with vision cone: non-reciprocal vs. reciprocal interactions
We study the behavior of the classical XY model on a two-dimensional square lattice, with interactions occurring within a vision cone of each spin. Via Monte Carlo simulations, we explore one non-reciprocal and two reciprocal implementations of these interactions. The corresponding energy involves couplings that depend non-trivially on the system's configuration, leading to both long-range and quasi-long-range ordered phases at low temperatures. Our results demonstrate that non-reciprocity is n…

This https://arxiv.org/abs/2505.19312 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIR_…

DocMMIR: A Framework for Document Multi-modal Information Retrieval
The rapid advancement of unsupervised representation learning and large-scale pre-trained vision-language models has significantly improved cross-modal retrieval tasks. However, existing multi-modal information retrieval (MMIR) studies lack a comprehensive exploration of document-level retrieval and suffer from the absence of cross-domain datasets at this granularity. To address this limitation, we introduce DocMMIR, a novel multi-modal document retrieval framework designed explicitly to unify …

Vision-Integrated High-Quality Neural Speech Coding
Yao Guo, Yang Ai, Rui-Chen Zheng, Hui-Peng Du, Xiao-Hang Jiang, Zhen-Hua Ling
https://arxiv.org/abs/2505.23379

Vision-Integrated High-Quality Neural Speech Coding
This paper proposes a novel vision-integrated neural speech codec (VNSC), which aims to enhance speech coding quality by leveraging visual modality information. In VNSC, the image analysis-synthesis module extracts visual features from lip images, while the feature fusion module facilitates interaction between the image analysis-synthesis module and the speech coding module, transmitting visual information to assist the speech coding process. Depending on whether visual information is available…

This https://arxiv.org/abs/2503.01879 has been replaced.
link: https://scholar.google.com/scholar?q=a

Nexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision
This work proposes an industry-level omni-modal large language model (LLM) pipeline that integrates auditory, visual, and linguistic modalities to overcome challenges such as limited tri-modal datasets, high computational costs, and complex feature alignments. Our pipeline consists of three main components: First, a modular framework enabling flexible configuration of various encoder-LLM-decoder architectures. Second, a lightweight training strategy that pre-trains audio-language alignment on t…

Refining Datapath for Microscaling ViTs
Can Xiao, Jianyi Cheng, Aaron Zhao
https://arxiv.org/abs/2505.22194 https://arxiv.org/pdf/250…

Refining Datapath for Microscaling ViTs
Vision Transformers (ViTs) leverage the transformer architecture to effectively capture global context, demonstrating strong performance in computer vision tasks. A major challenge in ViT hardware acceleration is that the model family contains complex arithmetic operations that are sensitive to model accuracy, such as the Softmax and LayerNorm operations, which cannot be mapped onto efficient hardware with low precision. Existing methods only exploit parallelism in the matrix multiplication ope…

STDR: Spatio-Temporal Decoupling for Real-Time Dynamic Scene Rendering
Zehao Li, Hao Jiang, Yujun Cai, Jianing Chen, Baolong Bi, Shuqin Gao, Honglong Zhao, Yiwei Wang, Tianlu Mao, Zhaoqi Wang
https://arxiv.org/abs/2505.22400

STDR: Spatio-Temporal Decoupling for Real-Time Dynamic Scene Rendering
Although dynamic scene reconstruction has long been a fundamental challenge in 3D vision, the recent emergence of 3D Gaussian Splatting (3DGS) offers a promising direction by enabling high-quality, real-time rendering through explicit Gaussian primitives. However, existing 3DGS-based methods for dynamic reconstruction often suffer from \textit{spatio-temporal incoherence} during initialization, where canonical Gaussians are constructed by aggregating observations from multiple frames without te…

This https://arxiv.org/abs/2505.13062 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csMM_…

Hearing from Silence: Reasoning Audio Descriptions from Silent Videos via Vision-Language Model
Humans can intuitively infer sounds from silent videos, but whether multimodal large language models can perform modal-mismatch reasoning without accessing target modalities remains relatively unexplored. Current text-assisted-video-to-audio (VT2A) methods excel in video foley tasks but struggle to acquire audio descriptions during inference. We introduce the task of Reasoning Audio Descriptions from Silent Videos (SVAD) to address this challenge and investigate vision-language models' (VLMs) c…

Mitigating Audiovisual Mismatch in Visual-Guide Audio Captioning
Le Xu, Chenxing Li, Yong Ren, Yujie Chen, Yu Gu, Ruibo Fu, Shan Yang, Dong Yu
https://arxiv.org/abs/2505.22045

Mitigating Audiovisual Mismatch in Visual-Guide Audio Captioning
Current vision-guided audio captioning systems frequently fail to address audiovisual misalignment in real-world scenarios, such as dubbed content or off-screen sounds. To bridge this critical gap, we present an entropy-aware gated fusion framework that dynamically modulates visual information flow through cross-modal uncertainty quantification. Our novel approach employs attention entropy analysis in cross-attention layers to automatically identify and suppress misleading visual cues during mo…