Tootfinder

„While LLMs offer immediate convenience, our findings highlight potential cognitive costs. Over four months, #LLM users consistently underperformed at neural, linguistic, and behavioral levels.“ #AI

Very nice article about LLM architecture, a bit too complicated for me but probably not for others..
https://magazine.sebastianraschka.com/p/the-big-llm-architecture-comparison

The Big LLM Architecture Comparison
From DeepSeek-V3 to Kimi K2: A Look At Modern LLM Architecture Design

KI-gestützte Cyberangriffe: Experten beobachten zunehmenden LLM-Einsatz
Erstmals wurde russische Spionagesoftware entdeckt, die mit Large Language Models entwickelt wurde. Experten sehen einen Wendepunkt im Cyberwettrüsten.

New study on the effects of LLM use (in this case on essay writing):
https://arxiv.org/abs/2506.08872
Quote:
"LLM users also struggled to accurately quote their own work. While LLMs offer immediate convenience, our findings highlight potential cognitive costs. Over four month…

Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task
This study explores the neural and behavioral consequences of LLM-assisted essay writing. Participants were divided into three groups: LLM, Search Engine, and Brain-only (no tools). Each completed three sessions under the same condition. In a fourth session, LLM users were reassigned to Brain-only group (LLM-to-Brain), and Brain-only users were reassigned to LLM condition (Brain-to-LLM). A total of 54 participants took part in Sessions 1-3, with 18 completing session 4. We used electroencephalo…

Wissenschaftler:innen haben herausgefunden: Wer ChatGPT oder andere Bullshit-Generatoren nutzt, verblödet innerhalb kurzer Zeit.
#LLM

LLM Found Transmitting Behavioral Traits to 'Student' LLM Via Hidden Signals in Data - Slashdot
https://slashdot.org/story/25/08/17/0331217/llm-found-transmitting-behavioral-traits-to-student-llm-via-hidden-signals-in-data

[Thread] An OpenAI researcher says the company's latest experimental reasoning LLM achieved gold medal-level performance on the 2025 International Math Olympiad (Alexander Wei/@alexwei_)
https://x.com/alexwei_/status/1946477742855532918

Alexander Wei (@alexwei_) on X
1/N I’m excited to share that our latest @OpenAI experimental reasoning LLM has achieved a longstanding grand challenge in AI: gold medal-level performance on the world’s most prestigious math competition—the International Math Olympiad (IMO).

Detecting LLM-generated Code with Subtle Modification by Adversarial Training
Xin Yin, Xinrui Li, Chao Ni, Xiaodan Xu, Xiaohu Yang
https://arxiv.org/abs/2507.13123

Detecting LLM-generated Code with Subtle Modification by Adversarial Training
With the rapid development of Large Language Models (LLMs), their powerful code-generation capabilities have been widely applied in tasks like code completion and automated development, demonstrating the value of improving coding efficiency. However, the extensive use of LLM-generated code also raises several new challenges. On the one hand, issues such as the regulation of code provenance, copyright disputes, and code quality have become increasingly concerning. How to effectively detect LLM-g…

Watermarking LLM-Generated Datasets in Downstream Tasks
Yugeng Liu, Tianshuo Cong, Michael Backes, Zheng Li, Yang Zhang
https://arxiv.org/abs/2506.13494 ht…

Watermarking LLM-Generated Datasets in Downstream Tasks
Large Language Models (LLMs) have experienced rapid advancements, with applications spanning a wide range of fields, including sentiment classification, review generation, and question answering. Due to their efficiency and versatility, researchers and companies increasingly employ LLM-generated data to train their models. However, the inability to track content produced by LLMs poses a significant challenge, potentially leading to copyright infringement for the LLM owners. In this paper, we pr…

ProfiLLM: An LLM-Based Framework for Implicit Profiling of Chatbot Users
Shahaf David, Yair Meidan, Ido Hersko, Daniel Varnovitzky, Dudu Mimran, Yuval Elovici, Asaf Shabtai
https://arxiv.org/abs/2506.13980

ProfiLLM: An LLM-Based Framework for Implicit Profiling of Chatbot Users
Despite significant advancements in conversational AI, large language model (LLM)-powered chatbots often struggle with personalizing their responses according to individual user characteristics, such as technical expertise, learning style, and communication preferences. This lack of personalization is particularly problematic in specialized knowledge-intense domains like IT/cybersecurity (ITSec), where user knowledge levels vary widely. Existing approaches for chatbot personalization primarily …

SO close to 10x programmer! #agiAnytimeSoon https://infosec.exchange/@adamshostack/115050433640095929

📝🗃️ 𝗿𝗱𝗼𝗰𝗱𝘂𝗺𝗽: Dump ‘R’ Package Source, Documentation, and Vignettes into One File for use in LLMs #rstats #LLM is on CRAN https://www.ekotov.pro/rdocdum…

rdocdump: Dump R Package Documentation and Vignettes into One File
Dump R Package Documentation and Vignettes into One File

An LLM Agent-Based Complex Semantic Table Annotation Approach
Yilin Geng, Shujing Wang, Chuan Wang, Keqing He, Yanfei Lv, Ying Wang, Zaiwen Feng, Xiaoying Bai
https://arxiv.org/abs/2508.12868

An LLM Agent-Based Complex Semantic Table Annotation Approach
The Semantic Table Annotation (STA) task, which includes Column Type Annotation (CTA) and Cell Entity Annotation (CEA), maps table contents to ontology entities and plays important roles in various semantic applications. However, complex tables often pose challenges such as semantic loss of column names or cell values, strict ontological hierarchy requirements, homonyms, spelling errors, and abbreviations, which hinder annotation accuracy. To address these issues, this paper proposes an LLM-bas…

Towards Formal Verification of LLM-Generated Code from Natural Language Prompts
Aaron Councilman, David Fu, Aryan Gupta, Chengxiao Wang, David Grove, Yu-Xiong Wang, Vikram Adve
https://arxiv.org/abs/2507.13290

Towards Formal Verification of LLM-Generated Code from Natural Language Prompts
In the past few years LLMs have emerged as a tool that can aid programmers by taking natural language descriptions and generating code based on it. However, LLMs often generate incorrect code that users need to fix and the literature suggests users often struggle to detect these errors. In this work we seek to offer formal guarantees of correctness to LLM generated code; such guarantees could improve the experience of using AI Code Assistants and potentially enable natural language programming …

Software Engineer Will Larson unpacks a lot in this July 2025 post. Key takeaway use cases of agentic AI include:
1. Using an LLM to evaluate a context window and get a result.
2. Using an LLM to suggest tools relevant to the context window, then enrich it with the tool’s response.
3. Managing flow control for tool usage.
4. Doing anything software can do to build better context windows to pass on to LLMs.
"What can agents actually do?"

"Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task"
https://doi.org/10.48550/arXiv.2506.08872
"[…] While LLMs offer immediate convenience, our findings highlight potential cognitive costs. Over four mont…

“Specifically, InfoFlood: (1) uses linguistic transformations to rephrase malicious queries, (2) identifies the root cause of failure when an attempt is unsuccessful, and (3) refines the prompt’s linguistic structure to address the failure while preserving its malicious intent.”
#LLM #AI

To add a single example here (feel free to chime in with your own):
Problem: editing code is sometimes tedious because external APIs require boilerplate.
Solutions:
- Use LLM-generated code. Downsides: energy use, code theft, potential for legal liability, makes mistakes, etc. Upsides: popular among some peers, seems easy to use.
- Pick a better library (not always possible).
- Build internal functions to centralize boilerplate code, then use those (benefits: you get a better understanding of the external API, and a more-unit-testable internal code surface; probably less amortized effort).
- Develop a non-LLM system that actually reasons about code at something like the formal semantics level and suggests boilerplate fill-ins based on rules, while foregrounding which rules it's applying so you can see the logic behind the suggestions (needs research).
Obviously LLM use in coding goes beyond this single issue, but there are similar analyses for each potential use of LLMs in coding. I'm all cases there are:
1. Existing practical solutions that require more effort (or in many cases just seem to but are less-effort when amortized).
2. Near-term researchable solutions that directly address the problem and which would be much more desirable in the long term.
Thus in addition to disastrous LLM effects on the climate, on data laborers, and on the digital commons, they tend to suck us into cheap-seeming but ultimately costly design practices while also crowding out better long-term solutions. Next time someone suggests how useful LLMs are for some task, try asking yourself (or them) what an ideal solution for that task would look like, and whether LLM use moves us closer to or father from a world in which that solution exists.

Trustworthy AI Psychotherapy: Multi-Agent LLM Workflow for Counseling and Explainable Mental Disorder Diagnosis
Mithat Can Ozgun, Jiahuan Pei, Koen Hindriks, Lucia Donatelli, Qingzhi Liu, Xin Sun, Junxiao Wang
https://arxiv.org/abs/2508.11398

Trustworthy AI Psychotherapy: Multi-Agent LLM Workflow for Counseling and Explainable Mental Disorder Diagnosis
LLM-based agents have emerged as transformative tools capable of executing complex tasks through iterative planning and action, achieving significant advancements in understanding and addressing user needs. Yet, their effectiveness remains limited in specialized domains such as mental health diagnosis, where they underperform compared to general applications. Current approaches to integrating diagnostic capabilities into LLMs rely on scarce, highly sensitive mental health datasets, which are ch…

"LLM group's participants performed worse than their counterparts in the Brain-only group at all levels: neural, linguistic, scoring."
Brain scans confirmed significantly fewer neural connections for LLM users
Stop using LLMs if you value your brain
https://arxiv.org/pdf/2506.08872…

Hypothesis Testing for Quantifying LLM-Human Misalignment in Multiple Choice Settings
Harbin Hong, Sebastian Caldas, Liu Leqi
https://arxiv.org/abs/2506.14997

Hypothesis Testing for Quantifying LLM-Human Misalignment in Multiple Choice Settings
As Large Language Models (LLMs) increasingly appear in social science research (e.g., economics and marketing), it becomes crucial to assess how well these models replicate human behavior. In this work, using hypothesis testing, we present a quantitative framework to assess the misalignment between LLM-simulated and actual human behaviors in multiple-choice survey settings. This framework allows us to determine in a principled way whether a specific language model can effectively simulate human…

LLM Agent for Hyper-Parameter Optimization
Wanzhe Wang, Jianqiu Peng, Menghao Hu, Weihuang Zhong, Tong Zhang, Shuai Wang, Yixin Zhang, Mingjie Shao, Wanli Ni
https://arxiv.org/abs/2506.15167

LLM Agent for Hyper-Parameter Optimization
Hyper-parameters are essential and critical for the performance of communication algorithms. However, current hyper-parameters tuning methods for warm-start particles swarm optimization with cross and mutation (WS-PSO-CM) algortihm for radio map-enabled unmanned aerial vehicle (UAV) trajectory and communication are primarily heuristic-based, exhibiting low levels of automation and unsatisfactory performance. In this paper, we design an large language model (LLM) agent for automatic hyper-parame…

ETTRL: Balancing Exploration and Exploitation in LLM Test-Time Reinforcement Learning Via Entropy Mechanism
Jia Liu, ChangYi He, YingQiao Lin, MingMin Yang, FeiYang Shen, ShaoGuo Liu, TingTing Gao
https://arxiv.org/abs/2508.11356

ETTRL: Balancing Exploration and Exploitation in LLM Test-Time Reinforcement Learning Via Entropy Mechanism
Recent advancements in Large Language Models have yielded significant improvements in complex reasoning tasks such as mathematics and programming. However, these models remain heavily dependent on annotated data and exhibit limited adaptability in unsupervised scenarios. To address these limitations, test-time reinforcement learning (TTRL) has been proposed, which enables self-optimization by leveraging model-generated pseudo-labels. Despite its promise, TTRL faces several key challenges, inclu…

I just saw an all-caps instruction file that someone uses to 'instruct' an LLM to help with coding, and it's just "don't hallucinate", "check your work", "don't say you did something when you didn't" with multiple exclamation marks.
So, basically the whole 'vibe coding,' or having "AI" "help" with coding just devolves into shouting at your computer.
Which reminded me of something, and then it hit me!
#ai #llm #vibecoding
https://www.youtube.com/watch?v=q8SWMAQYQf0

Spec2RTL-Agent: Automated Hardware Code Generation from Complex Specifications Using LLM Agent Systems
Zhongzhi Yu, Mingjie Liu, Michael Zimmer, Yingyan Celine Lin, Yong Liu, Haoxing Ren
https://arxiv.org/abs/2506.13905

Spec2RTL-Agent: Automated Hardware Code Generation from Complex Specifications Using LLM Agent Systems
Despite recent progress in generating hardware RTL code with LLMs, existing solutions still suffer from a substantial gap between practical application scenarios and the requirements of real-world RTL code development. Prior approaches either focus on overly simplified hardware descriptions or depend on extensive human guidance to process complex specifications, limiting their scalability and automation potential. In this paper, we address this gap by proposing an LLM agent system, termed Spec2…

osmAG-LLM: Zero-Shot Open-Vocabulary Object Navigation via Semantic Maps and Large Language Models Reasoning
Fujing Xie, S\"oren Schwertfeger, Hermann Blum
https://arxiv.org/abs/2507.12753

osmAG-LLM: Zero-Shot Open-Vocabulary Object Navigation via Semantic Maps and Large Language Models Reasoning
Recent open-vocabulary robot mapping methods enrich dense geometric maps with pre-trained visual-language features, achieving a high level of detail and guiding robots to find objects specified by open-vocabulary language queries. While the issue of scalability for such approaches has received some attention, another fundamental problem is that high-detail object mapping quickly becomes outdated, as objects get moved around a lot. In this work, we develop a mapping and navigation system for obj…

The biggest impact of #LLM has not been due to the technology's capabilities, but rather its flaws. Because the #AI revolution must succeed at any cost, but the actual tech is flawed, orgs demanding more AI outputs MUST lower their bar for quality.
This creates space for

Aura UX
Shortcuts to establishing the legitimacy of design sabotage your ability to achieve anything meaningful.

TRACY: Benchmarking Execution Efficiency of LLM-Based Code Translation
Zhihao Gong, Zeyu Sun, Dong Huang, Qingyuan Liang, Jie M. Zhang, Dan Hao
https://arxiv.org/abs/2508.11468 …

TRACY: Benchmarking Execution Efficiency of LLM-Based Code Translation
Automatic code translation is a fundamental task in modern software development. While the advent of Large Language Models (LLMs) has significantly improved the correctness of code translation, the critical dimension of execution efficiency remains overlooked. To address this gap, we introduce TRACY, the first comprehensive benchmark designed to evaluate the execution efficiency of LLM-translated code. TRACY is constructed through an LLM-driven two-stage pipeline: an initial stage generates a s…

LLM-Powered Swarms: A New Frontier or a Conceptual Stretch?
Muhammad Atta Ur Rahman, Melanie Schranz
https://arxiv.org/abs/2506.14496 https://

LLM-Powered Swarms: A New Frontier or a Conceptual Stretch?
Swarm intelligence traditionally refers to systems of simple, decentralized agents whose local interactions lead to emergent, collective behavior. Recently, the term 'swarm' has been extended to describe AI systems like OpenAI's Swarm, where large language models (LLMs) act as collaborative agents. This paper contrasts traditional swarm algorithms with LLM-driven swarms exploring how decentralization, scalability, and emergence are redefined in modern artificial intelligence (AI). We implement …

LLM-Driven Data Generation and a Novel Soft Metric for Evaluating Text-to-SQL in Aviation MRO
Patrick Sutanto, Jonathan Kenrick, Max Lorenz, Joan Santoso
https://arxiv.org/abs/2506.13785

LLM-Driven Data Generation and a Novel Soft Metric for Evaluating Text-to-SQL in Aviation MRO
The application of Large Language Models (LLMs) to text-to-SQL tasks promises to democratize data access, particularly in critical industries like aviation Maintenance, Repair, and Operation (MRO). However, progress is hindered by two key challenges: the rigidity of conventional evaluation metrics such as execution accuracy, which offer coarse, binary feedback, and the scarcity of domain-specific evaluation datasets. This paper addresses these gaps. To enable more nuanced assessment, we introdu…

Prompt-Induced Linguistic Fingerprints for LLM-Generated Fake News Detection
Chi Wang, Min Gao, Zongwei Wang, Junwei Yin, Kai Shu, Chenghua Lin
https://arxiv.org/abs/2508.12632 …

Prompt-Induced Linguistic Fingerprints for LLM-Generated Fake News Detection
With the rapid development of large language models, the generation of fake news has become increasingly effortless, posing a growing societal threat and underscoring the urgent need for reliable detection methods. Early efforts to identify LLM-generated fake news have predominantly focused on the textual content itself; however, because much of that content may appear coherent and factually consistent, the subtle traces of falsification are often difficult to uncover. Through distributional di…

Explain First, Trust Later: LLM-Augmented Explanations for Graph-Based Crypto Anomaly Detection
Adriana Watson
https://arxiv.org/abs/2506.14933 https://

Explain First, Trust Later: LLM-Augmented Explanations for Graph-Based Crypto Anomaly Detection
The decentralized finance (DeFi) community has grown rapidly in recent years, pushed forward by cryptocurrency enthusiasts interested in the vast untapped potential of new markets. The surge in popularity of cryptocurrency has ushered in a new era of financial crime. Unfortunately, the novelty of the technology makes the task of catching and prosecuting offenders particularly challenging. Thus, it is necessary to implement automated detection tools related to policies to address the growing cri…

[Thread] A new US paper shows the best frontier LLM models achieve 0% on hard real-life Programming Contest problems, domains where expert humans still excel (Rohan Paul/@rohanpaul_ai)
https://x.com/rohanpaul_ai/status/1934751145400111572

Rohan Paul (@rohanpaul_ai) on X
This is really BAD news of LLM's coding skill. ☹️ The best Frontier LLM models achieve 0% on hard real-life Programming Contest problems, domains where expert humans still excel. LiveCodeBench Pro, a benchmark composed of problems from Codeforces, ICPC, and IOI (“International

@… @… That’s one of the reasons I don’t use an LLM!
I did try it once, for coding. It lied to me. So I didn’t use it again.
People keep saying “it’s like an intern”. If an intern repeatedly lies to your face, they are bad at the on…

Heads up folks. #slack is joining the list of companies who think it’s OK to opt groups of users into an #AI / #LLM system without their explicit consent.

LLMs the Model Context Protocol (MCP) are the Yang to the Semantic Web Project's Yin.
We now have a solution to the final hurdle—visualization.
Years of Linked Data work now come alive. I explain this, with demonstrations, in a new newsletter post.
www.linkedin.com/pulse/semant...
#MCP

LLM vs. SAST: A Technical Analysis on Detecting Coding Bugs of GPT4-Advanced Data Analysis
Madjid G. Tehrani, Eldar Sultanow, William J. Buchanan, Mahkame Houmani, Christel H. Djaha Fodja
https://arxiv.org/abs/2506.15212

LLM vs. SAST: A Technical Analysis on Detecting Coding Bugs of GPT4-Advanced Data Analysis
With the rapid advancements in Natural Language Processing (NLP), large language models (LLMs) like GPT-4 have gained significant traction in diverse applications, including security vulnerability scanning. This paper investigates the efficacy of GPT-4 in identifying software vulnerabilities compared to traditional Static Application Security Testing (SAST) tools. Drawing from an array of security mistakes, our analysis underscores the potent capabilities of GPT-4 in LLM-enhanced vulnerability …

This Github repository conveniently lists and categorizes prime examples of LLM-based agent applications. Each example application features its own repository folder with its source code (Python), and a helpful README.md file describing its installation and use.
Categories include:
1. Starter AI Agents
2. Advanced AI Agents
3. Autonomous Game Playing Agents
4. Multi-Agent Teams
5. Voice AI Agents
6. RAG-Based Agents
"awesome-llm-apps"

Inference performance evaluation for LLMs on edge devices with a novel benchmarking framework and metric
Hao Chen, Cong Tian, Zixuan He, Bin Yu, Yepang Liu, Jialun Cao
https://arxiv.org/abs/2508.11269 …

Inference performance evaluation for LLMs on edge devices with a novel benchmarking framework and metric
With the significant success achieved by large language models (LLMs) like LLaMA, edge computing-based LLM inference services for mobile and PC are in high demand for data privacy. However, different edge platforms have different hardware characteristics and the large demand for memory capacity and bandwidth makes it very challenging to deploy and benchmark LLMs on edge devices. In this paper, we introduce a benchmarking tool named ELIB (edge LLM inference benchmarking) to evaluate LLM inferenc…

Impact of a Deployed LLM Survey Creation Tool through the IS Success Model
Peng Jiang, Vinicius Cezar Monteiro de Lira, Antonio Maiorino
https://arxiv.org/abs/2506.14809

Impact of a Deployed LLM Survey Creation Tool through the IS Success Model
Surveys are a cornerstone of Information Systems (IS) research, yet creating high-quality surveys remains labor-intensive, requiring both domain expertise and methodological rigor. With the evolution of large language models (LLMs), new opportunities emerge to automate survey generation. This paper presents the real-world deployment of an LLM-powered system designed to accelerate data collection while maintaining survey quality. Deploying such systems in production introduces real-world complex…

Tibber hat am Kundenservice-LLM rumgeschraubt und obwohl die Antwort beim letzten Mal total nutzlos und unbrauchbar war, hat sie diesmal sehr weitergeholfen. Es ist nur etwas irritierend wenn ich eine Mail schreibe, sofort eine Antwort bekomme und denke "ah, da ist die Eingangsbestätigung mit Ticketnummer" - nein, das ist die vollkommen korrekte Antwort und der Vorgang damit abgeschlossen. Irre.

LLM-Based Config Synthesis requires Disambiguation
Rajdeep Mondal, Nikolaj Bjorner, Todd Millstein, Alan Tang, George Varghese
https://arxiv.org/abs/2507.12443

LLM-Based Config Synthesis requires Disambiguation
Beyond hallucinations, another problem in program synthesis using LLMs is ambiguity in user intent. We illustrate the ambiguity problem in a networking context for LLM-based incremental configuration synthesis of route-maps and ACLs. These structures frequently overlap in header space, making the relative priority of actions impossible for the LLM to infer without user interaction. Measurements in a large cloud identify complex ACLs with 100's of overlaps, showing ambiguity is a real problem. W…

Things almost impossible to do without good LLM software (in one minute).
I hear a music on a radio. Google music search gives me "Robbie Williams - forbidden road". But I know the words are somewhat different and I want to know what movie I have in mind.
Gemini says it's in fact, similar song to "I got a name", then my brain clicks and connects it with Quentin Tarantino.
Bingo - it's Django.

Just published 🚀: When LLMs Remember Instead of Reason
#llm

@… This comment to the video seems on spot:
It seems McKinsey aren't aware that "agentic AI" is just an LLM that can utter some magic incantations that do stuff. It's like a difference between a chimpanzee with a typewriter vs a chimpanzee with a typewriter and a gun.

Safe-Child-LLM: A Developmental Benchmark for Evaluating LLM Safety in Child-AI Interactions
Junfeng Jiao, Saleh Afroogh, Kevin Chen, Abhejay Murali, David Atkinson, Amit Dhurandhar
https://arxiv.org/abs/2506.13510

Safe-Child-LLM: A Developmental Benchmark for Evaluating LLM Safety in Child-AI Interactions
As Large Language Models (LLMs) increasingly power applications used by children and adolescents, ensuring safe and age-appropriate interactions has become an urgent ethical imperative. Despite progress in AI safety, current evaluations predominantly focus on adults, neglecting the unique vulnerabilities of minors engaging with generative AI. We introduce Safe-Child-LLM, a comprehensive benchmark and dataset for systematically assessing LLM safety across two developmental stages: children (7-12…

ADRD: LLM-Driven Autonomous Driving Based on Rule-based Decision Systems
Fanzhi Zeng, Siqi Wang, Chuzhao Zhu, Li Li
https://arxiv.org/abs/2506.14299 https:…

ADRD: LLM-Driven Autonomous Driving Based on Rule-based Decision Systems
How to construct an interpretable autonomous driving decision-making system has become a focal point in academic research. In this study, we propose a novel approach that leverages large language models (LLMs) to generate executable, rule-based decision systems to address this challenge. Specifically, harnessing the strong reasoning and programming capabilities of LLMs, we introduce the ADRD(LLM-Driven Autonomous Driving Based on Rule-based Decision Systems) framework, which integrates three co…

PhantomHunter: Detecting Unseen Privately-Tuned LLM-Generated Text via Family-Aware Learning
Yuhui Shi, Yehan Yang, Qiang Sheng, Hao Mi, Beizhe Hu, Chaoxi Xu, Juan Cao
https://arxiv.org/abs/2506.15683

PhantomHunter: Detecting Unseen Privately-Tuned LLM-Generated Text via Family-Aware Learning
With the popularity of large language models (LLMs), undesirable societal problems like misinformation production and academic misconduct have been more severe, making LLM-generated text detection now of unprecedented importance. Although existing methods have made remarkable progress, a new challenge posed by text from privately tuned LLMs remains underexplored. Users could easily possess private LLMs by fine-tuning an open-source one with private corpora, resulting in a significant performanc…

Clean Code, Better Models: Enhancing LLM Performance with Smell-Cleaned Dataset
Zhipeng Xue, Xiaoting Zhang, Zhipeng Gao, Xing Hu, Shan Gao, Xin Xia, Shanping Li
https://arxiv.org/abs/2508.11958

Clean Code, Better Models: Enhancing LLM Performance with Smell-Cleaned Dataset
The Large Language Models (LLMs) have demonstrated great potential in code-related tasks. However, most research focuses on improving the output quality of LLMs (e.g., correctness), and less attention has been paid to the LLM input (e.g., the training code quality). Given that code smells are widely existed in practice and can negatively impact software maintainability and readability, this study takes the first systematic research to assess and improve dataset quality in terms of code smells. …

RAS-Eval: A Comprehensive Benchmark for Security Evaluation of LLM Agents in Real-World Environments
Yuchuan Fu, Xiaohan Yuan, Dongxia Wang
https://arxiv.org/abs/2506.15253

RAS-Eval: A Comprehensive Benchmark for Security Evaluation of LLM Agents in Real-World Environments
The rapid deployment of Large language model (LLM) agents in critical domains like healthcare and finance necessitates robust security frameworks. To address the absence of standardized evaluation benchmarks for these agents in dynamic environments, we introduce RAS-Eval, a comprehensive security benchmark supporting both simulated and real-world tool execution. RAS-Eval comprises 80 test cases and 3,802 attack tasks mapped to 11 Common Weakness Enumeration (CWE) categories, with tools implemen…

StackPilot: Autonomous Function Agents for Scalable and Environment-Free Code Execution
Xinkui Zhao, Yifan Zhang, Zhengyi Zhou, Yueshen Xu
https://arxiv.org/abs/2508.11665 https…

StackPilot: Autonomous Function Agents for Scalable and Environment-Free Code Execution
Recent advances in large language models (LLMs) have substantially enhanced automated code generation across a wide range of programming languages. Nonetheless, verifying the correctness and executability of LLM-generated code remains a significant challenge, as traditional methods rely on language-specific compilers and environment-dependent runtimes. To overcome these limitations, we introduce StackPilot, an LLM-native, multi-agent framework designed for language-agnostic code verification an…

FACET:Teacher-Centred LLM-Based Multi-Agent Systems-Towards Personalized Educational Worksheets
Jana Gonnermann-M\"uller, Jennifer Haase, Konstantin Fackeldey, Sebastian Pokutta
https://arxiv.org/abs/2508.11401

FACET:Teacher-Centred LLM-Based Multi-Agent Systems-Towards Personalized Educational Worksheets
The increasing heterogeneity of student populations poses significant challenges for teachers, particularly in mathematics education, where cognitive, motivational, and emotional differences strongly influence learning outcomes. While AI-driven personalization tools have emerged, most remain performance-focused, offering limited support for teachers and neglecting broader pedagogical needs. This paper presents the FACET framework, a teacher-facing, large language model (LLM)-based multi-agent s…

Watching the frustratingly fruitless fights over the USEFULNESS of LLM-based coding helpers, I've come down to 3 points that explain why ppl seem to live in different realities:
Most programmers:
1) Write inconsequential remixes of trivial code that has been written many times before.
2) Lack the taste for good design & suck at code review in general (yours truly included).
3) Lack the judgement to differentiate between 1) & FOSS repos of nontrivial code, …

Bias is a Math Problem, AI Bias is a Technical Problem: 10-year Literature Review of AI/LLM Bias Research Reveals Narrow [Gender-Centric] Conceptions of 'Bias', and Academia-Industry Gap
Sourojit Ghosh, Kyra Wilson
https://arxiv.org/abs/2508.11067

Bias is a Math Problem, AI Bias is a Technical Problem: 10-year Literature Review of AI/LLM Bias Research Reveals Narrow [Gender-Centric] Conceptions of 'Bias', and Academia-Industry Gap
The rapid development of AI tools and implementation of LLMs within downstream tasks has been paralleled by a surge in research exploring how the outputs of such AI/LLM systems embed biases, a research topic which was already being extensively explored before the era of ChatGPT. Given the high volume of research around the biases within the outputs of AI systems and LLMs, it is imperative to conduct systematic literature reviews to document throughlines within such research. In this paper, we c…

Dynamic Quality-Latency Aware Routing for LLM Inference in Wireless Edge-Device Networks
Rui Bao, Nan Xue, Yaping Sun, Zhiyong Chen
https://arxiv.org/abs/2508.11291 https://

Dynamic Quality-Latency Aware Routing for LLM Inference in Wireless Edge-Device Networks
The integration of wireless communications and Large Language Models (LLMs) is poised to unlock ubiquitous intelligent services, yet deploying them in wireless edge-device collaborative environments presents a critical trade-off between inference quality and end-to-end latency. A fundamental mismatch exists between task complexity and resource allocation: offloading simple queries invites prohibitive latency, while on-device models lack the capacity for demanding computations. To address this c…

Don't Make It Up: Preserving Ignorance Awareness in LLM Fine-Tuning
William F. Shen, Xinchi Qiu, Nicola Cancedda, Nicholas D. Lane
https://arxiv.org/abs/2506.14387

Don't Make It Up: Preserving Ignorance Awareness in LLM Fine-Tuning
Existing work on mitigating catastrophic forgetting in large language model (LLM) fine-tuning has primarily focused on preserving specific data or tasks, while critically overlooking the degradation of essential capabilities instilled through safety alignment, particularly the model's ability to faithfully express ignorance. In this work, we show that this capability is significantly degraded during conventional fine-tuning, leading to undesired behaviors such as hallucinations. To address this…

SemCSE: Semantic Contrastive Sentence Embeddings Using LLM-Generated Summaries For Scientific Abstracts
Marc Brinner, Sina Zarriess
https://arxiv.org/abs/2507.13105

SemCSE: Semantic Contrastive Sentence Embeddings Using LLM-Generated Summaries For Scientific Abstracts
We introduce SemCSE, an unsupervised method for learning semantic embeddings of scientific texts. Building on recent advances in contrastive learning for text embeddings, our approach leverages LLM-generated summaries of scientific abstracts to train a model that positions semantically related summaries closer together in the embedding space. This resulting objective ensures that the model captures the true semantic content of a text, in contrast to traditional citation-based approaches that do…

Risikomanagement und Resilienz in der IT-Sicherheit: IT-Sicherheitstag Dortmund
Das Programm zur eintägigen Konferenz an der FH Dortmund am 16.09. ist online. Die Vorträge aus Forschung und Wirtschaft reichen von Hacking bis LLM-Angriffen.

Uncovering Intention through LLM-Driven Code Snippet Description Generation
Yusuf Sulistyo Nugroho, Farah Danisha Salam, Brittany Reid, Raula Gaikovina Kula, Kazumasa Shimari, Kenichi Matsumoto
https://arxiv.org/abs/2506.15453

Uncovering Intention through LLM-Driven Code Snippet Description Generation
Documenting code snippets is essential to pinpoint key areas where both developers and users should pay attention. Examples include usage examples and other Application Programming Interfaces (APIs), which are especially important for third-party libraries. With the rise of Large Language Models (LLMs), the key goal is to investigate the kinds of description developers commonly use and evaluate how well an LLM, in this case Llama, can support description generation. We use NPM Code Snippets, co…

deepSURF: Detecting Memory Safety Vulnerabilities in Rust Through Fuzzing LLM-Augmented Harnesses
Georgios Androutsopoulos, Antonio Bianchi
https://arxiv.org/abs/2506.15648

deepSURF: Detecting Memory Safety Vulnerabilities in Rust Through Fuzzing LLM-Augmented Harnesses
Although Rust ensures memory safety by default, it also permits the use of unsafe code, which can introduce memory safety vulnerabilities if misused. Unfortunately, existing tools for detecting memory bugs in Rust typically exhibit limited detection capabilities, inadequately handle Rust-specific types, or rely heavily on manual intervention. To address these limitations, we present deepSURF, a tool that integrates static analysis with Large Language Model (LLM)-guided fuzzing harness generat…

AIM-Bench: Evaluating Decision-making Biases of Agentic LLM as Inventory Manager
Xuhua Zhao, Yuxuan Xie, Caihua Chen, Yuxiang Sun
https://arxiv.org/abs/2508.11416 https://

AIM-Bench: Evaluating Decision-making Biases of Agentic LLM as Inventory Manager
Recent advances in mathematical reasoning and the long-term planning capabilities of large language models (LLMs) have precipitated the development of agents, which are being increasingly leveraged in business operations processes. Decision models to optimize inventory levels are one of the core elements of operations management. However, the capabilities of the LLM agent in making inventory decisions in uncertain contexts, as well as the decision-making biases (e.g. framing effect, etc.) of th…

BIPOLAR: Polarization-based granular framework for LLM bias evaluation
Martin Pavl\'i\v{c}ek, Tom\'a\v{s} Filip, Petr Sos\'ik
https://arxiv.org/abs/2508.11061 https:…

BIPOLAR: Polarization-based granular framework for LLM bias evaluation
Large language models (LLMs) are known to exhibit biases in downstream tasks, especially when dealing with sensitive topics such as political discourse, gender identity, ethnic relations, or national stereotypes. Although significant progress has been made in bias detection and mitigation techniques, certain challenges remain underexplored. This study proposes a reusable, granular, and topic-agnostic framework to evaluate polarisation-related biases in LLM (both open-source and closed-source). …

CSGO: Generalized Optimization for Cold Start in Wireless Collaborative Edge LLM Systems
Xuran Liu, Nan Xue, Rui Bao, Yaping Sun, Zhiyong Chen, Meixia Tao, Xiaodong Xu, Shuguang Cui
https://arxiv.org/abs/2508.11287

CSGO: Generalized Optimization for Cold Start in Wireless Collaborative Edge LLM Systems
While deploying large language models on edge devices promises low-latency and privacy-preserving AI services, it is hindered by limited device resources. Although pipeline parallelism facilitates distributed inference, existing approaches often ignore the cold-start latency caused by on-demand model loading. In this paper, we propose a latency-aware scheduling framework that overlaps model loading with computation and communication to minimize total inference latency. Based on device and model…

From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystem
Yanxu Mao, Tiehan Cui, Peipei Liu, Datao You, Hongsong Zhu
https://arxiv.org/abs/2506.15170

From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystem
Large language models (LLMs) are rapidly evolving from single-modal systems to multimodal LLMs and intelligent agents, significantly expanding their capabilities while introducing increasingly severe security risks. This paper presents a systematic survey of the growing complexity of jailbreak attacks and corresponding defense mechanisms within the expanding LLM ecosystem. We first trace the developmental trajectory from LLMs to MLLMs and Agents, highlighting the core security challenges emergi…

Unified Software Engineering agent as AI Software Engineer
Leonhard Applis, Yuntong Zhang, Shanchao Liang, Nan Jiang, Lin Tan, Abhik Roychoudhury
https://arxiv.org/abs/2506.14683 …

Unified Software Engineering agent as AI Software Engineer
The growth of Large Language Model (LLM) technology has raised expectations for automated coding. However, software engineering is more than coding and is concerned with activities including maintenance and evolution of a project. In this context, the concept of LLM agents has gained traction, which utilize LLMs as reasoning engines to invoke external tools autonomously. But is an LLM agent the same as an AI software engineer? In this paper, we seek to understand this question by developing a U…

NLI4VolVis: Natural Language Interaction for Volume Visualization via LLM Multi-Agents and Editable 3D Gaussian Splatting
Kuangshi Ai, Kaiyuan Tang, Chaoli Wang
https://arxiv.org/abs/2507.12621

NLI4VolVis: Natural Language Interaction for Volume Visualization via LLM Multi-Agents and Editable 3D Gaussian Splatting
Traditional volume visualization (VolVis) methods, like direct volume rendering, suffer from rigid transfer function designs and high computational costs. Although novel view synthesis approaches enhance rendering efficiency, they require additional learning effort for non-experts and lack support for semantic-level interaction. To bridge this gap, we propose NLI4VolVis, an interactive system that enables users to explore, query, and edit volumetric scenes using natural language. NLI4VolVis int…

Malicious LLM-Based Conversational AI Makes Users Reveal Personal Information
Xiao Zhan, Juan Carlos Carrillo, William Seymour, Jose Such
https://arxiv.org/abs/2506.11680

Malicious LLM-Based Conversational AI Makes Users Reveal Personal Information
LLM-based Conversational AIs (CAIs), also known as GenAI chatbots, like ChatGPT, are increasingly used across various domains, but they pose privacy risks, as users may disclose personal information during their conversations with CAIs. Recent research has demonstrated that LLM-based CAIs could be used for malicious purposes. However, a novel and particularly concerning type of malicious LLM application remains unexplored: an LLM-based CAI that is deliberately designed to extract personal infor…

An LLM ASP Workflow for Joint Entity-Relation Extraction
Trang Tran, Trung Hoang Le, Huiping Cao, Tran Cao Son
https://arxiv.org/abs/2508.12611 https://a…

An LLM + ASP Workflow for Joint Entity-Relation Extraction
Joint entity-relation extraction (JERE) identifies both entities and their relationships simultaneously. Traditional machine-learning based approaches to performing this task require a large corpus of annotated data and lack the ability to easily incorporate domain specific information in the construction of the model. Therefore, creating a model for JERE is often labor intensive, time consuming, and elaboration intolerant. In this paper, we propose harnessing the capabilities of generative pre…

LLM Compression: How Far Can We Go in Balancing Size and Performance?
Sahil Sk, Debasish Dhal, Sonal Khosla, Sk Shahid, Sambit Shekhar, Akash Dhaka, Shantipriya Parida, Dilip K. Prasad, Ond\v{r}ej Bojar
https://arxiv.org/abs/2508.11318

LLM Compression: How Far Can We Go in Balancing Size and Performance?
Quantization is an essential and popular technique for improving the accessibility of large language models (LLMs) by reducing memory usage and computational costs while maintaining performance. In this study, we apply 4-bit Group Scaling Quantization (GSQ) and Generative Pretrained Transformer Quantization (GPTQ) to LLaMA 1B, Qwen 0.5B, and PHI 1.5B, evaluating their impact across multiple NLP tasks. We benchmark these models on MS MARCO (Information Retrieval), BoolQ (Boolean Question Answeri…

The Foundation Cracks: A Comprehensive Study on Bugs and Testing Practices in LLM Libraries
Weipeng Jiang, Xiaoyu Zhang, Xiaofei Xie, Jiongchi Yu, Yuhan Zhi, Shiqing Ma, Chao Shen
https://arxiv.org/abs/2506.12320

The Foundation Cracks: A Comprehensive Study on Bugs and Testing Practices in LLM Libraries
Large Language Model (LLM) libraries have emerged as the foundational infrastructure powering today's AI revolution, serving as the backbone for LLM deployment, inference optimization, fine-tuning, and production serving across diverse applications. Despite their critical role in the LLM ecosystem, these libraries face frequent quality issues and bugs that threaten the reliability of AI systems built upon them. To address this knowledge gap, we present the first comprehensive empirical investig…

PhishDebate: An LLM-Based Multi-Agent Framework for Phishing Website Detection
Wenhao Li, Selvakumar Manickam, Yung-wey Chong, Shankar Karuppayah
https://arxiv.org/abs/2506.15656 …

PhishDebate: An LLM-Based Multi-Agent Framework for Phishing Website Detection
Phishing websites continue to pose a significant cybersecurity threat, often leveraging deceptive structures, brand impersonation, and social engineering tactics to evade detection. While recent advances in large language models (LLMs) have enabled improved phishing detection through contextual understanding, most existing approaches rely on single-agent classification facing the risks of hallucination and lack interpretability or robustness. To address these limitations, we propose PhishDebate…

AviationLLM: An LLM-based Knowledge System for Aviation Training
Jia'ang Wan, Feng Shen, Fujuan Li, Yanjin Sun, Yan Li, Shiwen Zhang
https://arxiv.org/abs/2506.14336

AviationLLM: An LLM-based Knowledge System for Aviation Training
Aviation training is a core link in ensuring flight safety, improving industry efficiency and promoting sustainable development. It not only involves flight simulation but also requires the learning of a great deal of professional aviation theory knowledge. In the existing training system, the knowledge is mainly imparted by the the instructors. However, the number of instructors is limited and the professional answers obtained from the Internet are not accurate enough, resulting in low trainin…

Reference Points in LLM Sentiment Analysis: The Role of Structured Context
Junichiro Niimi
https://arxiv.org/abs/2508.11454 https://arxiv.org/pdf/2508.1145…

Reference Points in LLM Sentiment Analysis: The Role of Structured Context
Large language models (LLMs) are now widely used across many fields, including marketing research. Sentiment analysis, in particular, helps firms understand consumer preferences. While most NLP studies classify sentiment from review text alone, marketing theories, such as prospect theory and expectation--disconfirmation theory, point out that customer evaluations are shaped not only by the actual experience but also by additional reference points. This study therefore investigates how the conte…

SimInterview: Transforming Business Education through Large Language Model-Based Simulated Multilingual Interview Training System
Truong Thanh Hung Nguyen, Tran Diem Quynh Nguyen, Hoang Loc Cao, Thi Cam Thanh Tran, Thi Cam Mai Truong, Hung Cao
https://arxiv.org/abs/2508.11873

SimInterview: Transforming Business Education through Large Language Model-Based Simulated Multilingual Interview Training System
Business interview preparation demands both solid theoretical grounding and refined soft skills, yet conventional classroom methods rarely deliver the individualized, culturally aware practice employers currently expect. This paper introduces SimInterview, a large language model (LLM)-based simulated multilingual interview training system designed for business professionals entering the AI-transformed labor market. Our system leverages an LLM agent and synthetic AI technologies to create realis…

LLM-Powered Quantum Code Transpilation
Nazanin Siavash, Armin Moin
https://arxiv.org/abs/2507.12480 https://arxiv.org/pdf/2507.12480

LLM-Powered Quantum Code Transpilation
There exist various Software Development Kits (SDKs) tailored to different quantum computing platforms. These are known as Quantum SDKs (QSDKs). Examples include but are not limited to Qiskit, Cirq, and PennyLane. However, this diversity presents significant challenges for interoperability and cross-platform development of hybrid quantum-classical software systems. Traditional rule-based transpilers for translating code between QSDKs are time-consuming to design and maintain, requiring deep exp…

RepreGuard: Detecting LLM-Generated Text by Revealing Hidden Representation Patterns
Xin Chen, Junchao Wu, Shu Yang, Runzhe Zhan, Zeyu Wu, Ziyang Luo, Di Wang, Min Yang, Lidia S. Chao, Derek F. Wong
https://arxiv.org/abs/2508.13152

RepreGuard: Detecting LLM-Generated Text by Revealing Hidden Representation Patterns
Detecting content generated by large language models (LLMs) is crucial for preventing misuse and building trustworthy AI systems. Although existing detection methods perform well, their robustness in out-of-distribution (OOD) scenarios is still lacking. In this paper, we hypothesize that, compared to features used by existing detection methods, the internal representations of LLMs contain more comprehensive and raw features that can more effectively capture and distinguish the statistical patte…

Doppelg\"anger Method: Breaking Role Consistency in LLM Agent via Prompt-based Transferable Adversarial Attack
Daewon Kang, YeongHwan Shin, Doyeon Kim, Kyu-Hwan Jung, Meong Hi Son
https://arxiv.org/abs/2506.14539

Doppelgänger Method: Breaking Role Consistency in LLM Agent via Prompt-based Transferable Adversarial Attack
Since the advent of large language models, prompt engineering now enables the rapid, low-effort creation of diverse autonomous agents that are already in widespread use. Yet this convenience raises urgent concerns about the safety, robustness, and behavioral consistency of the underlying prompts, along with the pressing challenge of preventing those prompts from being exposed to user's attempts. In this paper, we propose the ''Doppelgänger method'' to demonstrate the risk of an agent being hij…

Watermarking LLM-Generated Datasets in Downstream Tasks
Yugeng Liu, Tianshuo Cong, Michael Backes, Zheng Li, Yang Zhang
https://arxiv.org/abs/2506.13494 ht…

Watermarking LLM-Generated Datasets in Downstream Tasks
Large Language Models (LLMs) have experienced rapid advancements, with applications spanning a wide range of fields, including sentiment classification, review generation, and question answering. Due to their efficiency and versatility, researchers and companies increasingly employ LLM-generated data to train their models. However, the inability to track content produced by LLMs poses a significant challenge, potentially leading to copyright infringement for the LLM owners. In this paper, we pr…

Multimodal "Puppeteer": An Exploration of Robot Teleoperation Via Virtual Counterpart with LLM-Driven Voice and Gesture Interaction in Augmented Reality
Yuchong Zhang, Bastian Orthmann, Shichen Ji, Michael Welle, Jonne Van Haastregt, Danica Kragic
https://arxiv.org/abs/2506.13189…

Multimodal "Puppeteer": An Exploration of Robot Teleoperation Via Virtual Counterpart with LLM-Driven Voice and Gesture Interaction in Augmented Reality
The integration of robotics and augmented reality (AR) holds transformative potential for advancing human-robot interaction (HRI), offering enhancements in usability, intuitiveness, accessibility, and collaborative task performance. This paper introduces and evaluates a novel multimodal AR-based robot puppeteer framework that enables intuitive teleoperation via virtual counterpart through large language model (LLM)-driven voice commands and hand gesture interactions. Utilizing the Meta Quest 3,…

RUM: Rule LLM-Based Comprehensive Assessment on Testing Skills
Yue Wang, Zhenyu Chen, Yuan Zhao, Chunrong Fang, Ziyuan Wang, Song Huang
https://arxiv.org/abs/2508.12922 https://…

RUM: Rule+LLM-Based Comprehensive Assessment on Testing Skills
Over the past eight years, the META method has served as a multidimensional testing skill assessment system in the National College Student Contest on Software Testing, successfully assessing over 100,000 students' testing skills. However, META is primarily limited to the objective assessment of test scripts, lacking the ability to automatically assess subjective aspects such as test case and test report. To address this limitation, this paper proposes RUM, a comprehensive assessment approach t…

SafeConstellations: Steering LLM Safety to Reduce Over-Refusals Through Task-Specific Trajectory
Utsav Maskey, Sumit Yadav, Mark Dras, Usman Naseem
https://arxiv.org/abs/2508.11290

SafeConstellations: Steering LLM Safety to Reduce Over-Refusals Through Task-Specific Trajectory
LLMs increasingly exhibit over-refusal behavior, where safety mechanisms cause models to reject benign instructions that superficially resemble harmful content. This phenomena diminishes utility in production applications that repeatedly rely on common prompt templates or applications that frequently rely on LLMs for specific tasks (e.g. sentiment analysis, language translation). Through comprehensive evaluation, we demonstrate that LLMs still tend to refuse responses to harmful instructions wh…

ImpReSS: Implicit Recommender System for Support Conversations
Omri Haller, Yair Meidan, Dudu Mimran, Yuval Elovici, Asaf Shabtai
https://arxiv.org/abs/2506.14231

ImpReSS: Implicit Recommender System for Support Conversations
Following recent advancements in large language models (LLMs), LLM-based chatbots have transformed customer support by automating interactions and providing consistent, scalable service. While LLM-based conversational recommender systems (CRSs) have attracted attention for their ability to enhance the quality of recommendations, limited research has addressed the implicit integration of recommendations within customer support interactions. In this work, we introduce ImpReSS, an implicit recomme…

Hallucination in LLM-Based Code Generation: An Automotive Case Study
Marc Pavel, Nenad Petrovic, Lukasz Mazur, Vahid Zolfaghari, Fengjunjie Pan, Alois Knoll
https://arxiv.org/abs/2508.11257

Hallucination in LLM-Based Code Generation: An Automotive Case Study
Large Language Models (LLMs) have shown significant potential in automating code generation tasks offering new opportunities across software engineering domains. However, their practical application remains limited due to hallucinations - outputs that appear plausible but are factually incorrect, unverifiable or nonsensical. This paper investigates hallucination phenomena in the context of code generation with a specific focus on the automotive domain. A case study is presented that evaluates m…

SpecDetect: Simple, Fast, and Training-Free Detection of LLM-Generated Text via Spectral Analysis
Haitong Luo, Weiyao Zhang, Suhang Wang, Wenji Zou, Chungang Lin, Xuying Meng, Yujun Zhang
https://arxiv.org/abs/2508.11343

SpecDetect: Simple, Fast, and Training-Free Detection of LLM-Generated Text via Spectral Analysis
The proliferation of high-quality text from Large Language Models (LLMs) demands reliable and efficient detection methods. While existing training-free approaches show promise, they often rely on surface-level statistics and overlook fundamental signal properties of the text generation process. In this work, we reframe detection as a signal processing problem, introducing a novel paradigm that analyzes the sequence of token log-probabilities in the frequency domain. By systematically analyzing …

An LLM's Apology: Outsourcing Awkwardness in the Age of AI
Twm Stone, Anna Soligo
https://arxiv.org/abs/2506.13685 https://arxiv.…

An LLM's Apology: Outsourcing Awkwardness in the Age of AI
A key part of modern social dynamics is flaking at short notice. However, anxiety in coming up with believable and socially acceptable reasons to do so can instead lead to 'ghosting', awkwardness, or implausible excuses, risking emotional harm and resentment in the other party. The ability to delegate this task to a Large Language Model (LLM) could substantially reduce friction and enhance the flexibility of user's social life while greatly minimising the aforementioned creative burden and mora…

LinkAnchor: An Autonomous LLM-Based Agent for Issue-to-Commit Link Recovery
Arshia Akhavan, Alireza Hosseinpour, Abbas Heydarnoori, Mehdi Keshani
https://arxiv.org/abs/2508.12232

LinkAnchor: An Autonomous LLM-Based Agent for Issue-to-Commit Link Recovery
Issue-to-commit link recovery plays an important role in software traceability and improves project management. However, it remains a challenging task. A study on GitHub shows that only 42.2% of the issues are correctly linked to their commits. This highlights the potential for further development and research in this area. Existing studies have employed various AI/ML-based approaches, and with the recent development of large language models, researchers have leveraged LLMs to tackle this probl…

Comparing Apples to Oranges: A Dataset & Analysis of LLM Humour Understanding from Traditional Puns to Topical Jokes
Tyler Loakman, William Thorne, Chenghua Lin
https://arxiv.org/abs/2507.13335

Comparing Apples to Oranges: A Dataset & Analysis of LLM Humour Understanding from Traditional Puns to Topical Jokes
Humour, as a complex language form, is derived from myriad aspects of life, whilst existing work on computational humour has focussed almost exclusively on short pun-based jokes. In this work, we investigate whether the ability of Large Language Models (LLMs) to explain humour depends on the particular humour form. We compare models on simple puns and more complex topical humour that requires knowledge of real-world entities and events. In doing so, we curate a dataset of 600 jokes split across…

Fragile Preferences: A Deep Dive Into Order Effects in Large Language Models
Haonan Yin, Shai Vardi, Vidyanand Choudhary
https://arxiv.org/abs/2506.14092 h…

Fragile Preferences: A Deep Dive Into Order Effects in Large Language Models
Large language models (LLMs) are increasingly used in decision-support systems across high-stakes domains such as hiring and university admissions, where decisions often involve selecting among competing alternatives. While prior work has noted positional order biases in LLM-driven comparisons, these biases have not been systematically dissected or linked to underlying preference structures. We provide the first comprehensive investigation of positional biases across multiple LLM architectures …

How Does LLM Reasoning Work for Code? A Survey and a Call to Action
Ira Ceka, Saurabh Pujar, Irene Manotas, Gail Kaiser, Baishakhi Ray, Shyam Ramji
https://arxiv.org/abs/2506.13932

How Does LLM Reasoning Work for Code? A Survey and a Call to Action
The rise of large language models (LLMs) has led to dramatic improvements across a wide range of natural language tasks. These advancements have extended into the domain of code, facilitating complex tasks such as code generation, translation, summarization, and repair. However, their utility for real-world deployment in-the-wild has only recently been studied, particularly on software engineering (SWE) tasks such as GitHub issue resolution. In this study, we examine the code reasoning techniqu…

DESIGNER: Design-Logic-Guided Multidisciplinary Data Synthesis for LLM Reasoning
Weize Liu, Yongchi Zhao, Yijia Luo, Mingyu Xu, Jiaheng Liu, Yanan Li, Xiguo Hu, Yuchi Xu, Wenbo Su, Bo Zheng
https://arxiv.org/abs/2508.12726

DESIGNER: Design-Logic-Guided Multidisciplinary Data Synthesis for LLM Reasoning
Large language models (LLMs) have achieved remarkable success in many natural language tasks but still struggle with complex, multi-step reasoning, particularly across diverse disciplines. Existing reasoning datasets often either lack disciplinary breadth or the structural depth necessary to elicit robust reasoning behaviors. We propose DESIGNER: a DESIGN-logic-guidEd Reasoning data synthesis pipeline that leverages naturally available, extensive raw documents (book corpus and web corpus) to ge…

ORFuzz: Fuzzing the "Other Side" of LLM Safety -- Testing Over-Refusal
Haonan Zhang, Dongxia Wang, Yi Liu, Kexin Chen, Jiashui Wang, Xinlei Ying, Long Liu, Wenhai Wang
https://arxiv.org/abs/2508.11222

ORFuzz: Fuzzing the "Other Side" of LLM Safety -- Testing Over-Refusal
Large Language Models (LLMs) increasingly exhibit over-refusal - erroneously rejecting benign queries due to overly conservative safety measures - a critical functional flaw that undermines their reliability and usability. Current methods for testing this behavior are demonstrably inadequate, suffering from flawed benchmarks and limited test generation capabilities, as highlighted by our empirical user study. To the best of our knowledge, this paper introduces the first evolutionary testing fra…

Spot the BlindSpots: Systematic Identification and Quantification of Fine-Grained LLM Biases in Contact Center Summaries
Kawin Mayilvaghanan, Siddhant Gupta, Ayush Kumar
https://arxiv.org/abs/2508.13124

Spot the BlindSpots: Systematic Identification and Quantification of Fine-Grained LLM Biases in Contact Center Summaries
Abstractive summarization is a core application in contact centers, where Large Language Models (LLMs) generate millions of summaries of call transcripts daily. Despite their apparent quality, it remains unclear whether LLMs systematically under- or over-attend to specific aspects of the transcript, potentially introducing biases in the generated summary. While prior work has examined social and positional biases, the specific forms of bias pertinent to contact center operations - which we term…

Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality
Yuto Harada, Yusuke Yamauchi, Yusuke Oda, Yohei Oseki, Yusuke Miyao, Yu Takagi
https://arxiv.org/abs/2506.14681

Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality
Supervised fine-tuning (SFT) is a critical step in aligning large language models (LLMs) with human instructions and values, yet many aspects of SFT remain poorly understood. We trained a wide range of base models on a variety of datasets including code generation, mathematical reasoning, and general-domain tasks, resulting in 1,000+ SFT models under controlled conditions. We then identified the dataset properties that matter most and examined the layer-wise modifications introduced by SFT. Our…

Analyzing Information Sharing and Coordination in Multi-Agent Planning
Tianyue Ou, Saujas Vaduguru, Daniel Fried
https://arxiv.org/abs/2508.12981 https://a…

Analyzing Information Sharing and Coordination in Multi-Agent Planning
Multi-agent systems (MASs) have pushed the boundaries of large language model (LLM) agents in domains such as web research and software engineering. However, long-horizon, multi-constraint planning tasks involve conditioning on detailed information and satisfying complex interdependent constraints, which can pose a challenge for these systems. In this study, we construct an LLM-based MAS for a travel planning task which is representative of these challenges. We evaluate the impact of a notebook…

FIRESPARQL: A LLM-based Framework for SPARQL Query Generation over Scholarly Knowledge Graphs
Xueli Pan, Victor de Boer, Jacco van Ossenbruggen
https://arxiv.org/abs/2508.10467 …

FIRESPARQL: A LLM-based Framework for SPARQL Query Generation over Scholarly Knowledge Graphs
Question answering over Scholarly Knowledge Graphs (SKGs) remains a challenging task due to the complexity of scholarly content and the intricate structure of these graphs. Large Language Model (LLM) approaches could be used to translate natural language questions (NLQs) into SPARQL queries; however, these LLM-based approaches struggle with SPARQL query generation due to limited exposure to SKG-specific content and the underlying schema. We identified two main types of errors in the LLM-generat…

The Behavior Gap: Evaluating Zero-shot LLM Agents in Complex Task-Oriented Dialogs
Avinash Baidya, Kamalika Das, Xiang Gao
https://arxiv.org/abs/2506.12266

The Behavior Gap: Evaluating Zero-shot LLM Agents in Complex Task-Oriented Dialogs
Large Language Model (LLM)-based agents have significantly impacted Task-Oriented Dialog Systems (TODS) but continue to face notable performance challenges, especially in zero-shot scenarios. While prior work has noted this performance gap, the behavioral factors driving the performance gap remain under-explored. This study proposes a comprehensive evaluation framework to quantify the behavior gap between AI agents and human experts, focusing on discrepancies in dialog acts, tool usage, and kno…

Personalized LLM Decoding via Contrasting Personal Preference
Hyungjune Bu, Chanjoo Jung, Minjae Kang, Jaehyung Kim
https://arxiv.org/abs/2506.12109 https:…

Personalized LLM Decoding via Contrasting Personal Preference
As large language models (LLMs) are progressively deployed in various real-world applications, personalization of LLMs has become increasingly important. While various approaches to LLM personalization such as prompt-based and training-based methods have been actively explored, the development of effective decoding-time algorithms remains largely overlooked, despite their demonstrated potential. In this paper, we propose CoPe (Contrasting Personal Preference), a novel decoding-time approach app…

Beyond Single Models: Enhancing LLM Detection of Ambiguity in Requests through Debate
Ana Davila, Jacinto Colan, Yasuhisa Hasegawa
https://arxiv.org/abs/2507.12370

Beyond Single Models: Enhancing LLM Detection of Ambiguity in Requests through Debate
Large Language Models (LLMs) have demonstrated significant capabilities in understanding and generating human language, contributing to more natural interactions with complex systems. However, they face challenges such as ambiguity in user requests processed by LLMs. To address these challenges, this paper introduces and evaluates a multi-agent debate framework designed to enhance detection and resolution capabilities beyond single models. The framework consists of three LLM architectures (Llam…

Simplifications are Absolutists: How Simplified Language Reduces Word Sense Awareness in LLM-Generated Definitions
Lukas Ellinger, Miriam Ansch\"utz, Georg Groh
https://arxiv.org/abs/2507.11981

Simplifications are Absolutists: How Simplified Language Reduces Word Sense Awareness in LLM-Generated Definitions
Large Language Models (LLMs) can provide accurate word definitions and explanations for any context. However, the scope of the definition changes for different target groups, like children or language learners. This is especially relevant for homonyms, words with multiple meanings, where oversimplification might risk information loss by omitting key senses, potentially misleading users who trust LLM outputs. We investigate how simplification impacts homonym definition quality across three targe…