Tootfinder

Feedstack: Layering Structured Representations over Unstructured Feedback to Scaffold Human AI Conversation
Hannah Vy Nguyen, Yu-Chun Grace Yen, Omar Shakir, Hang Huynh, Sebastian Gutierrez, June A. Smith, Sheila Jimenez, Salma Abdelgelil, Stephen MacNeil
https://arxiv.org/abs/2506.03052…

Feedstack: Layering Structured Representations over Unstructured Feedback to Scaffold Human AI Conversation
Many conversational user interfaces facilitate linear conversations with turn-based dialogue, similar to face-to-face conversations between people. However, digital conversations can afford more than simple back-and-forth; they can be layered with interaction techniques and structured representations that scaffold exploration, reflection, and shared understanding between users and AI systems. We introduce Feedstack, a speculative interface that augments feedback conversations with layered affor…

I left a comment last week on Smashing’s “Design Patterns For AI Interfaces” (which is still in moderation):
https://www.smashingmagazine.com/2025/07/design-patterns-ai-interfaces/
Essentially, if your UI doesn’t also try to prevent harm then it might …

Design Patterns For AI Interfaces — Smashing Magazine
Designing a new AI feature? Where do you even begin? Here’s a simple, practical overview with useful design patterns for better AI experiences.

Towards a unified user modeling language for engineering human centered AI systems
Aaron Conrardy, Alfredo Capozucca, Jordi Cabot
https://arxiv.org/abs/2505.24697

Towards a unified user modeling language for engineering human centered AI systems
In today's digital society, personalization has become a crucial aspect of software applications, significantly impacting user experience and engagement. A new wave of intelligent user interfaces, such as AI-based conversational agents, has the potential to enable such personalization beyond what other types of interfaces could offer in the past. Personalization requires the ability to specify a complete user profile, covering as many dimensions as possible, such as potential accessibility cons…

DeformTune: A Deformable XAI Music Prototype for Non-Musicians
Ziqing Xu, Nick Bryan-Kinns
https://arxiv.org/abs/2508.00160 https://arxiv.org/pdf/2508.0016…

DeformTune: A Deformable XAI Music Prototype for Non-Musicians
Many existing AI music generation tools rely on text prompts, complex interfaces, or instrument-like controls, which may require musical or technical knowledge that non-musicians do not possess. This paper introduces DeformTune, a prototype system that combines a tactile deformable interface with the MeasureVAE model to explore more intuitive, embodied, and explainable AI interaction. We conducted a preliminary study with 11 adult participants without formal musical training to investigate thei…

Agent Network Protocol Technical White Paper
Gaowei Chang, Eidan Lin, Chengxuan Yuan, Rizhao Cai, Binbin Chen, Xuan Xie, Yin Zhang
https://arxiv.org/abs/2508.00007 https://

Agent Network Protocol Technical White Paper
With the development of large models and autonomous decision-making AI, agents are rapidly becoming the new entities of the internet, following mobile apps. However, existing internet infrastructure is primarily designed for human interaction, creating data silos, unfriendly interfaces, and high collaboration costs among agents, making it difficult to support the needs for large-scale agent interconnection and collaboration. The internet is undergoing a profound transformation, showing four cor…

Beyond Rigid AI: Towards Natural Human-Machine Symbiosis for Interoperative Surgical Assistance
Lalithkumar Seenivasan, Jiru Xu, Roger D. Soberanis Mukul, Hao Ding, Grayson Byrd, Yu-Chun Ku, Jose L. Porras, Masaru Ishii, Mathias Unberath
https://arxiv.org/abs/2507.23088

Beyond Rigid AI: Towards Natural Human-Machine Symbiosis for Interoperative Surgical Assistance
Emerging surgical data science and robotics solutions, especially those designed to provide assistance in situ, require natural human-machine interfaces to fully unlock their potential in providing adaptive and intuitive aid. Contemporary AI-driven solutions remain inherently rigid, offering limited flexibility and restricting natural human-machine interaction in dynamic surgical environments. These solutions rely heavily on extensive task-specific pre-training, fixed object categories, and exp…

From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows
Mohamed Amine Ferrag, Norbert Tihanyi, Djallel Hamouda, Leandros Maglaras, Merouane Debbah
https://arxiv.org/abs/2506.23260

From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows
Autonomous AI agents powered by large language models (LLMs) with structured function-calling interfaces have dramatically expanded capabilities for real-time data retrieval, complex computation, and multi-step orchestration. Yet, the explosive proliferation of plugins, connectors, and inter-agent protocols has outpaced discovery mechanisms and security practices, resulting in brittle integrations vulnerable to diverse threats. In this survey, we introduce the first unified, end-to-end threat m…

Sources: Meta is in advanced talks to buy PlayAI, which uses AI to replicate voices for interfaces that are as "responsive as a conversation between two people" (Bloomberg)
https://www.bloomberg.com/news/articles/20

Semantic-aware Digital Twin for AI-based CSI Acquisition
Jiajia Guo, Yiming Cui, Shi Jin
https://arxiv.org/abs/2506.21126 https://arx…

Semantic-aware Digital Twin for AI-based CSI Acquisition
Artificial intelligence (AI) substantially enhances channel state information (CSI) acquisition performance but is limited by its reliance on single-modality information and deployment challenges, particularly in dataset collection. This paper investigates the use of semantic-aware digital twin (DT) to enhance AI-based CSI acquisition. We first briefly introduce the motivation and recent advancements in AI-driven CSI acquisition and semantic-aware DT employment for air interfaces. Then, we thor…

Replaced article(s) found for cs.AI. https://arxiv.org/list/cs.AI/new
[2/4]:
- Visual Grounding Methods for Efficient Interaction with Desktop Graphical User Interfaces
El Hassane Ettifouri, Jessica L\'opez Espejel, Laura Minkova, Tassnim Dardouri, Walid Dahhane

Email as the Interface to Generative AI Models: Seamless Administrative Automation
Andres Navarro, Carlos de Quinto, Jos\'e Alberto Hern\'andez
https://arxiv.org/abs/2506.23850

Email as the Interface to Generative AI Models: Seamless Administrative Automation
This paper introduces a novel architectural framework that integrates Large Language Models (LLMs) with email interfaces to automate administrative tasks, specifically targeting accessibility barriers in enterprise environments. The system connects email communication channels with Optical Character Recognition (OCR) and intelligent automation, enabling non-technical administrative staff to delegate complex form-filling and document processing tasks using familiar email interfaces. By treating …

This https://arxiv.org/abs/2505.15799 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCY_…

The Agentic Economy
Generative AI has transformed human-computer interaction by enabling natural language interfaces and the emergence of autonomous agents capable of acting on users' behalf. While early applications have improved individual productivity, these gains have largely been confined to predefined tasks within existing workflows. We argue that the more profound economic impact lies in reducing communication frictions between consumers and businesses. This shift could reorganize markets, redistribute powe…

An advanced AI driven database system
M. Tedeschi, S. Rizwan, C. Shringi, V. Devram Chandgir, S. Belich
https://arxiv.org/abs/2507.17778 https://arxiv.org/…

An advanced AI driven database system
Contemporary database systems, while effective, suffer severe issues related to complexity and usability, especially among individuals who lack technical expertise but are unfamiliar with query languages like Structured Query Language (SQL). This paper presents a new database system supported by Artificial Intelligence (AI), which is intended to improve the management of data using natural language processing (NLP) - based intuitive interfaces, and automatic creation of structured queries and s…

GamerAstra: Enhancing Video Game Accessibility for Blind and Low-Vision Players through a Multi-Agent AI Framework
Tianrun Qiu, Changxin Chen, Sizhe Cheng, Yiming Yang, Yixiao Guo, Zhicong Lu, Yuxin Ma
https://arxiv.org/abs/2506.22937

GamerAstra: Enhancing Video Game Accessibility for Blind and Low-Vision Players through a Multi-Agent AI Framework
Blind and low-vision (BLV) players encounter critical challenges in engaging with video games due to the inaccessibility of visual elements, difficulties in navigating interfaces, and limitations in sending interaction input. Moreover, the development of specialized accessibility features typically requires substantial programming effort and is often implemented on a game-by-game basis. To address these challenges, we introduce \textit{GamerAstra}, a generalized accessibility framework that lev…

Fiduciary AI for the Future of Brain-Technology Interactions
Abhishek Bhattacharjee, Jack Pilkington, Nita Farahany
https://arxiv.org/abs/2507.14339 https:…

Fiduciary AI for the Future of Brain-Technology Interactions
Brain foundation models represent a new frontier in AI: instead of processing text or images, these models interpret real-time neural signals from EEG, fMRI, and other neurotechnologies. When integrated with brain-computer interfaces (BCIs), they may enable transformative applications-from thought controlled devices to neuroprosthetics-by interpreting and acting on brain activity in milliseconds. However, these same systems pose unprecedented risks, including the exploitation of subconscious ne…

FPGA & VPU Co-Processing in Space Applications: Development and Testing with DSP/AI Benchmarks
Vasileios Leon, Charalampos Bezaitis, George Lentaris, Dimitrios Soudris, Dionysios Reisis, Elissaios-Alexios Papatheofanous, Angelos Kyriakos, Aubrey Dunne, Arne Samuelsson, David Steenari
https://arxiv.org/abs/2506.12968

FPGA & VPU Co-Processing in Space Applications: Development and Testing with DSP/AI Benchmarks
The advent of computationally demanding algorithms and high data rate instruments in new space applications pushes the space industry to explore disruptive solutions for on-board data processing. We examine heterogeneous computing architectures involving high-performance and low-power commercial SoCs. The current paper implements an FPGA with VPU co-processing architecture utilizing the CIF & LCD interfaces for I/O data transfers. A Kintex FPGA serves as our framing processor and heritage accel…

User-Centered Design with AI in the Loop: A Case Study of Rapid User Interface Prototyping with "Vibe Coding"
Tianyi Li, Tanay Maheshwari, Alex Voelker
https://arxiv.org/abs/2507.21012

User-Centered Design with AI in the Loop: A Case Study of Rapid User Interface Prototyping with "Vibe Coding"
We present a case study of using generative user interfaces, or ``vibe coding,'' a method leveraging large language models (LLMs) for generating code via natural language prompts, to support rapid prototyping in user-centered design (UCD). Extending traditional UCD practices, we propose an AI-in-the-loop ideate-prototyping process. We share insights from an empirical experience integrating this process to develop an interactive data analytics interface for highway traffic engineers to effective…

What to expect at WWDC 2025: iOS 26 and other software branding changes, redesigned software interfaces, productivity enhancements, minor AI changes, and more (Mark Gurman/Bloomberg)
https://www.bloomberg.com/news/articles/20

Design Patterns of Human-AI Interfaces in Healthcare
Rui Sheng, Chuhan Shi, Sobhan Lotfi, Shiyi Liu, Adam Perer, Huamin Qu, Furui Cheng
https://arxiv.org/abs/2507.12721

Design Patterns of Human-AI Interfaces in Healthcare
Human-AI interfaces play a crucial role in advancing practices and research within the healthcare domain. However, designing such interfaces presents a substantial challenge for designers. In this paper, we propose systematic guidance for designing human-AI interfaces in typical healthcare scenarios by summarizing the design patterns for presenting and interacting with common information entities. To deepen our understanding of these 12 design patterns, we interviewed 12 healthcare professional…

Correlative angstrom-scale microscopy and spectroscopy of graphite-water interfaces
Lalith Krishna Samanth Bonagiri, Diana M. Arvelo, Fujia Zhao, Jaehyeon Kim, Qian Ai, Shan Zhou, Kaustubh S. Panse, Ricardo Garcia, Yingjie Zhang
https://arxiv.org/abs/2506.09908

Correlative angstrom-scale microscopy and spectroscopy of graphite-water interfaces
Water at solid surfaces is key for many processes ranging from biological signal transduction to membrane separation and renewable energy conversion. However, under realistic conditions, which often include environmental and surface charge variations, the interfacial water structure remains elusive. Here we overcome this limit by combining three-dimensional atomic force microscopy and interface-sensitive Raman spectroscopy to characterize the graphite-water interfacial structure in situ. Throug…

CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding
Yuchen Zhou, Jiamin Wu, Zichen Ren, Zhouheng Yao, Weiheng Lu, Kunyu Peng, Qihao Zheng, Chunfeng Song, Wanli Ouyang, Chao Gou
https://arxiv.org/abs/2506.23075

CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding
Understanding and decoding brain activity from electroencephalography (EEG) signals is a fundamental challenge in neuroscience and AI, with applications in cognition, emotion recognition, diagnosis, and brain-computer interfaces. While recent EEG foundation models advance generalized decoding via unified architectures and large-scale pretraining, they adopt a scale-agnostic dense modeling paradigm inherited from NLP and vision. This design neglects a core property of neural activity: cross-scal…

Real-Time AI-Driven Pipeline for Automated Medical Study Content Generation in Low-Resource Settings: A Kenyan Case Study
Emmanuel Korir, Eugene Wechuli
https://arxiv.org/abs/2507.05212

Real-Time AI-Driven Pipeline for Automated Medical Study Content Generation in Low-Resource Settings: A Kenyan Case Study
Juvenotes is a real-time AI-driven pipeline that automates the transformation of academic documents into structured exam-style question banks, optimized for low-resource medical education settings in Kenya. The system combines Azure Document Intelligence for OCR and Azure AI Foundry (OpenAI o3-mini) for question and answer generation in a microservices architecture, with a Vue/TypeScript frontend and AdonisJS backend. Mobile-first design, bandwidth-sensitive interfaces, institutional tagging, a…

Research and Analysis of Employers' Opinion on the Necessary Skills that Students in the Field of Web Programming Should Possess
Yordan Kalmukov
https://arxiv.org/abs/2506.11084

Research and Analysis of Employers' Opinion on the Necessary Skills that Students in the Field of Web Programming Should Possess
In the era of artificial intelligence (AI) and chatbots, based on large language models that can generate programming code in any language, write texts and summarize information, it is obvious that the requirements of employers for graduating students have already changed. The modern IT world offers significant automation of programming through software frameworks and a huge set of third-party libraries and application programming interfaces (APIs). All these tools provide most of the necessary…

IoT and Older Adults: Towards Multimodal EMG and AI-Based Interaction with Smart Home
Wies{\l}aw Kope\'c, Jaros{\l}aw Kowalski, Aleksander Majda, Anna Duszyk-Bogorodzka, Anna Jaskulska, Cezary Biele
https://arxiv.org/abs/2507.19479

IoT and Older Adults: Towards Multimodal EMG and AI-Based Interaction with Smart Home
We report preliminary insights from an exploratory study on non-standard non-invasive interfaces for Smart Home Technologies (SHT). This study is part of a broader research project on effective Smart Home ecosystem Sagacity that will target older adults, impaired persons, and other groups disadvantaged in the main technology discourse. Therefore, this research is in line with a long-term research framework of the HASE research group (Human Aspects in Science and Engineering) by the Living Lab K…

On using AI for EEG-based BCI applications: problems, current challenges and future trends
Thomas Barbera, Jacopo Burger, Alessandro D'Amelio, Simone Zini, Simone Bianco, Raffaella Lanzarotti, Paolo Napoletano, Giuseppe Boccignone, Jose Luis Contreras-Vidal
https://arxiv.org/abs/2506.16168

On using AI for EEG-based BCI applications: problems, current challenges and future trends
Imagine unlocking the power of the mind to communicate, create, and even interact with the world around us. Recent breakthroughs in Artificial Intelligence (AI), especially in how machines "see" and "understand" language, are now fueling exciting progress in decoding brain signals from scalp electroencephalography (EEG). Prima facie, this opens the door to revolutionary brain-computer interfaces (BCIs) designed for real life, moving beyond traditional uses to envision Brain-to-Speech, Brain-to-…

Prompt Migration: Stabilizing GenAI Applications with Evolving Large Language Models
Shivani Tripathi, Pushpanjali Nema, Aditya Halder, Shi Qiao, Alekh Jindal
https://arxiv.org/abs/2507.05573

Prompt Migration: Stabilizing GenAI Applications with Evolving Large Language Models
Generative AI is transforming business applications by enabling natural language interfaces and intelligent automation. However, the underlying large language models (LLMs) are evolving rapidly and so prompting them consistently is a challenge. This leads to inconsistent and unpredictable application behavior, undermining the reliability that businesses require for mission-critical workflows. In this paper, we introduce the concept of prompt migration as a systematic approach to stabilizing Gen…

Do Conversational Interfaces Limit Creativity? Exploring Visual Graph Systems for Creative Writing
Abhinav Sood, Maria Teresa Llano, Jon McCormack
https://arxiv.org/abs/2507.08260

Do Conversational Interfaces Limit Creativity? Exploring Visual Graph Systems for Creative Writing
We present a graphical, node-based system through which users can visually chain generative AI models for creative tasks. Research in the area of chaining LLMs has found that while chaining provides transparency, controllability and guardrails to approach certain tasks, chaining with pre-defined LLM steps prevents free exploration. Using cognitive processes from creativity research as a basis, we create a system that addresses the inherent constraints of chat-based AI interactions. Specifically…

Conceptualization, Operationalization, and Measurement of Machine Companionship: A Scoping Review
Jaime Banks, Zhixin Li
https://arxiv.org/abs/2506.18119 h…

Conceptualization, Operationalization, and Measurement of Machine Companionship: A Scoping Review
The notion of machine companions has long been embedded in social-technological imaginaries. Recent advances in AI have moved those media musings into believable sociality manifested in interfaces, robotic bodies, and devices. Those machines are often referred to colloquially as "companions" yet there is little careful engagement of machine companionship (MC) as a formal concept or measured variable. This PRISMA-guided scoping review systematically samples, surveys, and synthesizes current scho…

WebXAII: an open-source web framework to study human-XAI interaction
Jules Leguy, Pierre-Antoine Jean, Felipe Torres Figueroa, S\'ebastien Harispe
https://arxiv.org/abs/2506.14777

WebXAII: an open-source web framework to study human-XAI interaction
This article introduces WebXAII, an open-source web framework designed to facilitate research on human interaction with eXplainable Artificial Intelligence (XAI) systems. The field of XAI is rapidly expanding, driven by the growing societal implications of the widespread adoption of AI (and in particular machine learning) across diverse applications. Researchers who study the interaction between humans and XAI techniques typically develop ad hoc interfaces in order to conduct their studies. The…