Tootfinder

I left a comment last week on Smashing’s “Design Patterns For AI Interfaces” (which is still in moderation):
https://www.smashingmagazine.com/2025/07/design-patterns-ai-interfaces/
Essentially, if your UI doesn’t also try to prevent harm then it might …

Design Patterns For AI Interfaces — Smashing Magazine
Designing a new AI feature? Where do you even begin? Here’s a simple, practical overview with useful design patterns for better AI experiences.

Towards a unified user modeling language for engineering human centered AI systems
Aaron Conrardy, Alfredo Capozucca, Jordi Cabot
https://arxiv.org/abs/2505.24697

Towards a unified user modeling language for engineering human centered AI systems
In today's digital society, personalization has become a crucial aspect of software applications, significantly impacting user experience and engagement. A new wave of intelligent user interfaces, such as AI-based conversational agents, has the potential to enable such personalization beyond what other types of interfaces could offer in the past. Personalization requires the ability to specify a complete user profile, covering as many dimensions as possible, such as potential accessibility cons…

Beyond Rigid AI: Towards Natural Human-Machine Symbiosis for Interoperative Surgical Assistance
Lalithkumar Seenivasan, Jiru Xu, Roger D. Soberanis Mukul, Hao Ding, Grayson Byrd, Yu-Chun Ku, Jose L. Porras, Masaru Ishii, Mathias Unberath
https://arxiv.org/abs/2507.23088

Beyond Rigid AI: Towards Natural Human-Machine Symbiosis for Interoperative Surgical Assistance
Emerging surgical data science and robotics solutions, especially those designed to provide assistance in situ, require natural human-machine interfaces to fully unlock their potential in providing adaptive and intuitive aid. Contemporary AI-driven solutions remain inherently rigid, offering limited flexibility and restricting natural human-machine interaction in dynamic surgical environments. These solutions rely heavily on extensive task-specific pre-training, fixed object categories, and exp…

From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows
Mohamed Amine Ferrag, Norbert Tihanyi, Djallel Hamouda, Leandros Maglaras, Merouane Debbah
https://arxiv.org/abs/2506.23260

From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows
Autonomous AI agents powered by large language models (LLMs) with structured function-calling interfaces have dramatically expanded capabilities for real-time data retrieval, complex computation, and multi-step orchestration. Yet, the explosive proliferation of plugins, connectors, and inter-agent protocols has outpaced discovery mechanisms and security practices, resulting in brittle integrations vulnerable to diverse threats. In this survey, we introduce the first unified, end-to-end threat m…

Email as the Interface to Generative AI Models: Seamless Administrative Automation
Andres Navarro, Carlos de Quinto, Jos\'e Alberto Hern\'andez
https://arxiv.org/abs/2506.23850

Email as the Interface to Generative AI Models: Seamless Administrative Automation
This paper introduces a novel architectural framework that integrates Large Language Models (LLMs) with email interfaces to automate administrative tasks, specifically targeting accessibility barriers in enterprise environments. The system connects email communication channels with Optical Character Recognition (OCR) and intelligent automation, enabling non-technical administrative staff to delegate complex form-filling and document processing tasks using familiar email interfaces. By treating …

GamerAstra: Enhancing Video Game Accessibility for Blind and Low-Vision Players through a Multi-Agent AI Framework
Tianrun Qiu, Changxin Chen, Sizhe Cheng, Yiming Yang, Yixiao Guo, Zhicong Lu, Yuxin Ma
https://arxiv.org/abs/2506.22937

GamerAstra: Enhancing Video Game Accessibility for Blind and Low-Vision Players through a Multi-Agent AI Framework
Blind and low-vision (BLV) players encounter critical challenges in engaging with video games due to the inaccessibility of visual elements, difficulties in navigating interfaces, and limitations in sending interaction input. Moreover, the development of specialized accessibility features typically requires substantial programming effort and is often implemented on a game-by-game basis. To address these challenges, we introduce \textit{GamerAstra}, a generalized accessibility framework that lev…

AI use cases introduced by Rob Finn from EMBL-EBI, as pointed out in his #CORDI2025 keynote "Delivering life science data resources in a world of growing data and impacts from AI"
https://www.nfdi.de/cordi-2025/keynotes/

Sources: Meta is in advanced talks to buy PlayAI, which uses AI to replicate voices for interfaces that are as "responsive as a conversation between two people" (Bloomberg)
https://www.bloomberg.com/news/articles/20

Semantic-aware Digital Twin for AI-based CSI Acquisition
Jiajia Guo, Yiming Cui, Shi Jin
https://arxiv.org/abs/2506.21126 https://arx…

Semantic-aware Digital Twin for AI-based CSI Acquisition
Artificial intelligence (AI) substantially enhances channel state information (CSI) acquisition performance but is limited by its reliance on single-modality information and deployment challenges, particularly in dataset collection. This paper investigates the use of semantic-aware digital twin (DT) to enhance AI-based CSI acquisition. We first briefly introduce the motivation and recent advancements in AI-driven CSI acquisition and semantic-aware DT employment for air interfaces. Then, we thor…

AI as IA: The use and abuse of artificial intelligence (AI) for human enhancement through intellectual augmentation (IA)
Alexandre Erler, Vincent C. M\"uller
https://arxiv.org/abs/2508.16642

AI as IA: The use and abuse of artificial intelligence (AI) for human enhancement through intellectual augmentation (IA)
This paper offers an overview of the prospects and ethics of using AI to achieve human enhancement, and more broadly what we call intellectual augmentation (IA). After explaining the central notions of human enhancement, IA, and AI, we discuss the state of the art in terms of the main technologies for IA, with or without brain-computer interfaces. Given this picture, we discuss potential ethical problems, namely inadequate performance, safety, coercion and manipulation, privacy, cognitive liber…

CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding
Yuchen Zhou, Jiamin Wu, Zichen Ren, Zhouheng Yao, Weiheng Lu, Kunyu Peng, Qihao Zheng, Chunfeng Song, Wanli Ouyang, Chao Gou
https://arxiv.org/abs/2506.23075

CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding
Understanding and decoding brain activity from electroencephalography (EEG) signals is a fundamental challenge in neuroscience and AI, with applications in cognition, emotion recognition, diagnosis, and brain-computer interfaces. While recent EEG foundation models advance generalized decoding via unified architectures and large-scale pretraining, they adopt a scale-agnostic dense modeling paradigm inherited from NLP and vision. This design neglects a core property of neural activity: cross-scal…

Replaced article(s) found for cs.AI. https://arxiv.org/list/cs.AI/new
[2/4]:
- Visual Grounding Methods for Efficient Interaction with Desktop Graphical User Interfaces
El Hassane Ettifouri, Jessica L\'opez Espejel, Laura Minkova, Tassnim Dardouri, Walid Dahhane

An advanced AI driven database system
M. Tedeschi, S. Rizwan, C. Shringi, V. Devram Chandgir, S. Belich
https://arxiv.org/abs/2507.17778 https://arxiv.org/…

An advanced AI driven database system
Contemporary database systems, while effective, suffer severe issues related to complexity and usability, especially among individuals who lack technical expertise but are unfamiliar with query languages like Structured Query Language (SQL). This paper presents a new database system supported by Artificial Intelligence (AI), which is intended to improve the management of data using natural language processing (NLP) - based intuitive interfaces, and automatic creation of structured queries and s…

Interface on demand: Towards AI native Control interfaces for 6G
Abhishek Dandekar, Prashiddha D. Thapa, Ashrafur Rahman, Julius Schulz-Zander
https://arxiv.org/abs/2508.15595 h…

Interface on demand: Towards AI native Control interfaces for 6G
Traditional standardized network interfaces face significant limitations, including vendor-specific incompatibilities, rigid design assumptions, and lack of adaptability for new functionalities. We propose a multi-agent framework leveraging large language models (LLMs) to generate control interfaces on demand between network functions (NFs). This includes a matching agent, which aligns required control functionalities with NF capabilities, and a code-generation agent, which generates the necess…

Beyond the Bermuda Triangle of Contention: IOMMU Interference in Mixed Criticality Systems
Diogo Costa, Jose Martins, Sandro Pinto
https://arxiv.org/abs/2508.19670 https://

Beyond the Bermuda Triangle of Contention: IOMMU Interference in Mixed Criticality Systems
As Mixed Criticality Systems (MCSs) evolve, they increasingly integrate heterogeneous computing platforms, combining general-purpose processors with specialized accelerators such as AI engines, GPUs, and high-speed networking interfaces. This heterogeneity introduces challenges, as these accelerators and DMA-capable devices act as independent bus masters, directly accessing memory. Consequently, ensuring both security and timing predictability in such environments becomes critical. To address t…

User-Centered Design with AI in the Loop: A Case Study of Rapid User Interface Prototyping with "Vibe Coding"
Tianyi Li, Tanay Maheshwari, Alex Voelker
https://arxiv.org/abs/2507.21012

User-Centered Design with AI in the Loop: A Case Study of Rapid User Interface Prototyping with "Vibe Coding"
We present a case study of using generative user interfaces, or ``vibe coding,'' a method leveraging large language models (LLMs) for generating code via natural language prompts, to support rapid prototyping in user-centered design (UCD). Extending traditional UCD practices, we propose an AI-in-the-loop ideate-prototyping process. We share insights from an empirical experience integrating this process to develop an interactive data analytics interface for highway traffic engineers to effective…

Fiduciary AI for the Future of Brain-Technology Interactions
Abhishek Bhattacharjee, Jack Pilkington, Nita Farahany
https://arxiv.org/abs/2507.14339 https:…

Fiduciary AI for the Future of Brain-Technology Interactions
Brain foundation models represent a new frontier in AI: instead of processing text or images, these models interpret real-time neural signals from EEG, fMRI, and other neurotechnologies. When integrated with brain-computer interfaces (BCIs), they may enable transformative applications-from thought controlled devices to neuroprosthetics-by interpreting and acting on brain activity in milliseconds. However, these same systems pose unprecedented risks, including the exploitation of subconscious ne…

FPGA & VPU Co-Processing in Space Applications: Development and Testing with DSP/AI Benchmarks
Vasileios Leon, Charalampos Bezaitis, George Lentaris, Dimitrios Soudris, Dionysios Reisis, Elissaios-Alexios Papatheofanous, Angelos Kyriakos, Aubrey Dunne, Arne Samuelsson, David Steenari
https://arxiv.org/abs/2506.12968

FPGA & VPU Co-Processing in Space Applications: Development and Testing with DSP/AI Benchmarks
The advent of computationally demanding algorithms and high data rate instruments in new space applications pushes the space industry to explore disruptive solutions for on-board data processing. We examine heterogeneous computing architectures involving high-performance and low-power commercial SoCs. The current paper implements an FPGA with VPU co-processing architecture utilizing the CIF & LCD interfaces for I/O data transfers. A Kintex FPGA serves as our framing processor and heritage accel…

webMCP: Efficient AI-Native Client-Side Interaction for Agent-Ready Web Design
D. Perera
https://arxiv.org/abs/2508.09171 https://arxiv.org/pdf/2508.09171

webMCP: Efficient AI-Native Client-Side Interaction for Agent-Ready Web Design
Current AI agents create significant barriers for users by requiring extensive processing to understand web pages, making AI-assisted web interaction slow and expensive. This paper introduces webMCP (Web Machine Context & Procedure), a client-side standard that embeds structured interaction metadata directly into web pages, enabling more efficient human-AI collaboration on existing websites. webMCP transforms how AI agents understand web interfaces by providing explicit mappings between page el…

What to expect at WWDC 2025: iOS 26 and other software branding changes, redesigned software interfaces, productivity enhancements, minor AI changes, and more (Mark Gurman/Bloomberg)
https://www.bloomberg.com/news/articles/20

IoT and Older Adults: Towards Multimodal EMG and AI-Based Interaction with Smart Home
Wies{\l}aw Kope\'c, Jaros{\l}aw Kowalski, Aleksander Majda, Anna Duszyk-Bogorodzka, Anna Jaskulska, Cezary Biele
https://arxiv.org/abs/2507.19479

IoT and Older Adults: Towards Multimodal EMG and AI-Based Interaction with Smart Home
We report preliminary insights from an exploratory study on non-standard non-invasive interfaces for Smart Home Technologies (SHT). This study is part of a broader research project on effective Smart Home ecosystem Sagacity that will target older adults, impaired persons, and other groups disadvantaged in the main technology discourse. Therefore, this research is in line with a long-term research framework of the HASE research group (Human Aspects in Science and Engineering) by the Living Lab K…

Design Patterns of Human-AI Interfaces in Healthcare
Rui Sheng, Chuhan Shi, Sobhan Lotfi, Shiyi Liu, Adam Perer, Huamin Qu, Furui Cheng
https://arxiv.org/abs/2507.12721

Design Patterns of Human-AI Interfaces in Healthcare
Human-AI interfaces play a crucial role in advancing practices and research within the healthcare domain. However, designing such interfaces presents a substantial challenge for designers. In this paper, we propose systematic guidance for designing human-AI interfaces in typical healthcare scenarios by summarizing the design patterns for presenting and interacting with common information entities. To deepen our understanding of these 12 design patterns, we interviewed 12 healthcare professional…

Correlative angstrom-scale microscopy and spectroscopy of graphite-water interfaces
Lalith Krishna Samanth Bonagiri, Diana M. Arvelo, Fujia Zhao, Jaehyeon Kim, Qian Ai, Shan Zhou, Kaustubh S. Panse, Ricardo Garcia, Yingjie Zhang
https://arxiv.org/abs/2506.09908

Correlative angstrom-scale microscopy and spectroscopy of graphite-water interfaces
Water at solid surfaces is key for many processes ranging from biological signal transduction to membrane separation and renewable energy conversion. However, under realistic conditions, which often include environmental and surface charge variations, the interfacial water structure remains elusive. Here we overcome this limit by combining three-dimensional atomic force microscopy and interface-sensitive Raman spectroscopy to characterize the graphite-water interfacial structure in situ. Throug…

Towards Embodied Agentic AI: Review and Classification of LLM- and VLM-Driven Robot Autonomy and Interaction
Sahar Salimpour, Lei Fu, Farhad Keramat, Leonardo Militano, Giovanni Toffetti, Harry Edelman, Jorge Pe\~na Queralta
https://arxiv.org/abs/2508.05294

Towards Embodied Agentic AI: Review and Classification of LLM- and VLM-Driven Robot Autonomy and Interaction
Foundation models, including large language models (LLMs) and vision-language models (VLMs), have recently enabled novel approaches to robot autonomy and human-robot interfaces. In parallel, vision-language-action models (VLAs) or large behavior models (BLMs) are increasing the dexterity and capabilities of robotic systems. This survey paper focuses on those words advancing towards agentic applications and architectures. This includes initial efforts exploring GPT-style interfaces to tooling, a…

FROGENT: An End-to-End Full-process Drug Design Agent
Qihua Pan, Dong Xu, Jenna Xinyi Yao, Lijia Ma, Zexuan Zhu, Junkai Ji
https://arxiv.org/abs/2508.10760 https://

FROGENT: An End-to-End Full-process Drug Design Agent
Powerful AI tools for drug discovery reside in isolated web apps, desktop programs, and code libraries. Such fragmentation forces scientists to manage incompatible interfaces and specialized scripts, which can be a cumbersome and repetitive process. To address this issue, a Full-pROcess druG dEsign ageNT, named FROGENT, has been proposed. Specifically, FROGENT utilizes a Large Language Model and the Model Context Protocol to integrate multiple dynamic biochemical databases, extensible tool libr…

On using AI for EEG-based BCI applications: problems, current challenges and future trends
Thomas Barbera, Jacopo Burger, Alessandro D'Amelio, Simone Zini, Simone Bianco, Raffaella Lanzarotti, Paolo Napoletano, Giuseppe Boccignone, Jose Luis Contreras-Vidal
https://arxiv.org/abs/2506.16168

On using AI for EEG-based BCI applications: problems, current challenges and future trends
Imagine unlocking the power of the mind to communicate, create, and even interact with the world around us. Recent breakthroughs in Artificial Intelligence (AI), especially in how machines "see" and "understand" language, are now fueling exciting progress in decoding brain signals from scalp electroencephalography (EEG). Prima facie, this opens the door to revolutionary brain-computer interfaces (BCIs) designed for real life, moving beyond traditional uses to envision Brain-to-Speech, Brain-to-…

Real-Time AI-Driven Pipeline for Automated Medical Study Content Generation in Low-Resource Settings: A Kenyan Case Study
Emmanuel Korir, Eugene Wechuli
https://arxiv.org/abs/2507.05212

Real-Time AI-Driven Pipeline for Automated Medical Study Content Generation in Low-Resource Settings: A Kenyan Case Study
Juvenotes is a real-time AI-driven pipeline that automates the transformation of academic documents into structured exam-style question banks, optimized for low-resource medical education settings in Kenya. The system combines Azure Document Intelligence for OCR and Azure AI Foundry (OpenAI o3-mini) for question and answer generation in a microservices architecture, with a Vue/TypeScript frontend and AdonisJS backend. Mobile-first design, bandwidth-sensitive interfaces, institutional tagging, a…

An Automated Multi-Modal Evaluation Framework for Mobile Intelligent Assistants
Meiping Wang, Jian Zhong, Rongduo Han, Liming Kang, Zhengkun Shi, Xiao Liang, Xing Lin, Nan Gao, Haining Zhang
https://arxiv.org/abs/2508.09507

An Automated Multi-Modal Evaluation Framework for Mobile Intelligent Assistants
With the rapid development of mobile intelligent assistant technologies, multi-modal AI assistants have become essential interfaces for daily user interactions. However, current evaluation methods face challenges including high manual costs, inconsistent standards, and subjective bias. This paper proposes an automated multi-modal evaluation framework based on large language models and multi-agent collaboration. The framework employs a three-tier agent architecture consisting of interaction eval…

Research and Analysis of Employers' Opinion on the Necessary Skills that Students in the Field of Web Programming Should Possess
Yordan Kalmukov
https://arxiv.org/abs/2506.11084

Research and Analysis of Employers' Opinion on the Necessary Skills that Students in the Field of Web Programming Should Possess
In the era of artificial intelligence (AI) and chatbots, based on large language models that can generate programming code in any language, write texts and summarize information, it is obvious that the requirements of employers for graduating students have already changed. The modern IT world offers significant automation of programming through software frameworks and a huge set of third-party libraries and application programming interfaces (APIs). All these tools provide most of the necessary…

Imposing AI: Deceptive design patterns against sustainability
Ana\"elle Beignon, Thomas Thibault, Nolwenn Maudet
https://arxiv.org/abs/2508.08672 https://

Imposing AI: Deceptive design patterns against sustainability
Generative AI is being massively deployed in digital services, at a scale that will result in significant environmental harm. We document how tech companies are transforming established user interfaces to impose AI use and show how and to what extent these strategies fit within established deceptive pattern categories. We identify two main design strategies that are implemented to impose AI use in both personal and professional contexts: imposing AI features in interfaces at the expense of exis…

Prompt Migration: Stabilizing GenAI Applications with Evolving Large Language Models
Shivani Tripathi, Pushpanjali Nema, Aditya Halder, Shi Qiao, Alekh Jindal
https://arxiv.org/abs/2507.05573

Prompt Migration: Stabilizing GenAI Applications with Evolving Large Language Models
Generative AI is transforming business applications by enabling natural language interfaces and intelligent automation. However, the underlying large language models (LLMs) are evolving rapidly and so prompting them consistently is a challenge. This leads to inconsistent and unpredictable application behavior, undermining the reliability that businesses require for mission-critical workflows. In this paper, we introduce the concept of prompt migration as a systematic approach to stabilizing Gen…

Conceptualization, Operationalization, and Measurement of Machine Companionship: A Scoping Review
Jaime Banks, Zhixin Li
https://arxiv.org/abs/2506.18119 h…

Conceptualization, Operationalization, and Measurement of Machine Companionship: A Scoping Review
The notion of machine companions has long been embedded in social-technological imaginaries. Recent advances in AI have moved those media musings into believable sociality manifested in interfaces, robotic bodies, and devices. Those machines are often referred to colloquially as "companions" yet there is little careful engagement of machine companionship (MC) as a formal concept or measured variable. This PRISMA-guided scoping review systematically samples, surveys, and synthesizes current scho…

Do Conversational Interfaces Limit Creativity? Exploring Visual Graph Systems for Creative Writing
Abhinav Sood, Maria Teresa Llano, Jon McCormack
https://arxiv.org/abs/2507.08260

Do Conversational Interfaces Limit Creativity? Exploring Visual Graph Systems for Creative Writing
We present a graphical, node-based system through which users can visually chain generative AI models for creative tasks. Research in the area of chaining LLMs has found that while chaining provides transparency, controllability and guardrails to approach certain tasks, chaining with pre-defined LLM steps prevents free exploration. Using cognitive processes from creativity research as a basis, we create a system that addresses the inherent constraints of chat-based AI interactions. Specifically…

Agent Network Protocol Technical White Paper
Gaowei Chang, Eidan Lin, Chengxuan Yuan, Rizhao Cai, Binbin Chen, Xuan Xie, Yin Zhang
https://arxiv.org/abs/2508.00007 https://

Agent Network Protocol Technical White Paper
With the development of large models and autonomous decision-making AI, agents are rapidly becoming the new entities of the internet, following mobile apps. However, existing internet infrastructure is primarily designed for human interaction, creating data silos, unfriendly interfaces, and high collaboration costs among agents, making it difficult to support the needs for large-scale agent interconnection and collaboration. The internet is undergoing a profound transformation, showing four cor…

Feedstack: Layering Structured Representations over Unstructured Feedback to Scaffold Human AI Conversation
Hannah Vy Nguyen, Yu-Chun Grace Yen, Omar Shakir, Hang Huynh, Sebastian Gutierrez, June A. Smith, Sheila Jimenez, Salma Abdelgelil, Stephen MacNeil
https://arxiv.org/abs/2506.03052…

Feedstack: Layering Structured Representations over Unstructured Feedback to Scaffold Human AI Conversation
Many conversational user interfaces facilitate linear conversations with turn-based dialogue, similar to face-to-face conversations between people. However, digital conversations can afford more than simple back-and-forth; they can be layered with interaction techniques and structured representations that scaffold exploration, reflection, and shared understanding between users and AI systems. We introduce Feedstack, a speculative interface that augments feedback conversations with layered affor…

WebXAII: an open-source web framework to study human-XAI interaction
Jules Leguy, Pierre-Antoine Jean, Felipe Torres Figueroa, S\'ebastien Harispe
https://arxiv.org/abs/2506.14777

WebXAII: an open-source web framework to study human-XAI interaction
This article introduces WebXAII, an open-source web framework designed to facilitate research on human interaction with eXplainable Artificial Intelligence (XAI) systems. The field of XAI is rapidly expanding, driven by the growing societal implications of the widespread adoption of AI (and in particular machine learning) across diverse applications. Researchers who study the interaction between humans and XAI techniques typically develop ad hoc interfaces in order to conduct their studies. The…

CognitiveArm: Enabling Real-Time EEG-Controlled Prosthetic Arm Using Embodied Machine Learning
Abdul Basit, Maha Nawaz, Saim Rehman, Muhammad Shafique
https://arxiv.org/abs/2508.07731

CognitiveArm: Enabling Real-Time EEG-Controlled Prosthetic Arm Using Embodied Machine Learning
Efficient control of prosthetic limbs via non-invasive brain-computer interfaces (BCIs) requires advanced EEG processing, including pre-filtering, feature extraction, and action prediction, performed in real time on edge AI hardware. Achieving this on resource-constrained devices presents challenges in balancing model complexity, computational efficiency, and latency. We present CognitiveArm, an EEG-driven, brain-controlled prosthetic system implemented on embedded AI hardware, achieving real-t…

DeformTune: A Deformable XAI Music Prototype for Non-Musicians
Ziqing Xu, Nick Bryan-Kinns
https://arxiv.org/abs/2508.00160 https://arxiv.org/pdf/2508.0016…

DeformTune: A Deformable XAI Music Prototype for Non-Musicians
Many existing AI music generation tools rely on text prompts, complex interfaces, or instrument-like controls, which may require musical or technical knowledge that non-musicians do not possess. This paper introduces DeformTune, a prototype system that combines a tactile deformable interface with the MeasureVAE model to explore more intuitive, embodied, and explainable AI interaction. We conducted a preliminary study with 11 adult participants without formal musical training to investigate thei…