Tootfinder

"We simply don’t know to defend against these attacks. We have zero agentic AI systems that are secure against these attacks.[…] It’s an existential problem that, near as I can tell, most people developing these technologies are just pretending isn’t there."
https://www.

How do Humans and LLMs Process Confusing Code?
Youssef Abdelsalam, Norman Peitek, Anna-Maria Maurer, Mariya Toneva, Sven Apel
https://arxiv.org/abs/2508.18547 https://

How do Humans and LLMs Process Confusing Code?
Already today, humans and programming assistants based on large language models (LLMs) collaborate in everyday programming tasks. Clearly, a misalignment between how LLMs and programmers comprehend code can lead to misunderstandings, inefficiencies, low code quality, and bugs. A key question in this space is whether humans and LLMs are confused by the same kind of code. This would not only guide our choices of integrating LLMs in software engineering workflows, but also inform about possible …

Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs
Yao Fu, Xianxuan Long, Runchao Li, Haotian Yu, Mu Sheng, Xiaotian Han, Yu Yin, Pan Li
https://arxiv.org/abs/2508.19432

Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs
Quantization enables efficient deployment of large language models (LLMs) in resource-constrained environments by significantly reducing memory and computation costs. While quantized LLMs often maintain performance on perplexity and zero-shot tasks, their impact on truthfulness-whether generating truthful or deceptive responses-remains largely unexplored. In this work, we introduce TruthfulnessEval, a comprehensive evaluation framework for assessing the truthfulness of quantized LLMs across thr…

A Case Study on the Effectiveness of LLMs in Verification with Proof Assistants
Bar{\i}\c{s} Bayaz{\i}t, Yao Li, Xujie Si
https://arxiv.org/abs/2508.18587 https://

A Case Study on the Effectiveness of LLMs in Verification with Proof Assistants
Large language models (LLMs) can potentially help with verification using proof assistants by automating proofs. However, it is unclear how effective LLMs are in this task. In this paper, we perform a case study based on two mature Rocq projects: the hs-to-coq tool and Verdi. We evaluate the effectiveness of LLMs in generating proofs by both quantitative and qualitative analysis. Our study finds that: (1) external dependencies and context in the same source file can significantly help proof gen…

Grammatikfehler machen Prompt Injections wahrscheinlicher
LLMs sind anfälliger für Prompt Injections oder einfaches Übergehen der Leitplanken, wenn man Fehler im Prompt macht.
https://www.

Smooth Reading: Bridging the Gap of Recurrent LLM to Self-Attention LLM on Long-Context Tasks
Kai Liu, Zhan Su, Peijie Dong, Fengran Mo, Jianfei Gao, ShaoTing Zhang, Kai Chen
https://arxiv.org/abs/2507.19353

Smooth Reading: Bridging the Gap of Recurrent LLM to Self-Attention LLM on Long-Context Tasks
Recently, recurrent large language models (Recurrent LLMs) with linear computational complexity have re-emerged as efficient alternatives to self-attention-based LLMs (Self-Attention LLMs), which have quadratic complexity. However, Recurrent LLMs often underperform on long-context tasks due to their limited fixed-size memory. Previous research has primarily focused on enhancing the memory capacity of Recurrent LLMs through architectural innovations, but these approaches have not yet enabled Rec…

LLMs exhibit "potemkin understanding"!
Hope the methodology here is better than the last LLM-hater arxiv paper that came through
Must read it more carefully...
https://mathstodon.xyz/@gregeganSF/114758840374128081

Greg Egan (@gregeganSF@mathstodon.xyz)
Attached: 1 image “Potemkin Understanding in Large Language Models” A detailed analysis of the incoherent application of concepts by LLMs, showing how benchmarks that reliably establish domain competence in humans can be passed by LLMs lacking similar competence. H/T @acowley@mastodon.social Link: https://arxiv.org/abs/2506.21521

LLMs in the SOC: An Empirical Study of Human-AI Collaboration in Security Operations Centres
Ronal Singh, Shahroz Tariq, Fatemeh Jalalvand, Mohan Baruwal Chhetri, Surya Nepal, Cecile Paris, Martin Lochner
https://arxiv.org/abs/2508.18947

LLMs in the SOC: An Empirical Study of Human-AI Collaboration in Security Operations Centres
The integration of Large Language Models (LLMs) into Security Operations Centres (SOCs) presents a transformative, yet still evolving, opportunity to reduce analyst workload through human-AI collaboration. However, their real-world application in SOCs remains underexplored. To address this gap, we present a longitudinal study of 3,090 analyst queries from 45 SOC analysts over 10 months. Our analysis reveals that analysts use LLMs as on-demand aids for sensemaking and context-building, rather th…

Fine-Tuning LLMs For ‘Good’ Behavior Makes Them More Likely To Say No https://www.404media.co/fine-tuning-llms-cognitive-bias-decision-making-study/

Should LLMs be WEIRD? Exploring WEIRDness and Human Rights in Large Language Models
Ke Zhou, Marios Constantinides, Daniele Quercia
https://arxiv.org/abs/2508.19269 https://

Should LLMs be WEIRD? Exploring WEIRDness and Human Rights in Large Language Models
Large language models (LLMs) are often trained on data that reflect WEIRD values: Western, Educated, Industrialized, Rich, and Democratic. This raises concerns about cultural bias and fairness. Using responses to the World Values Survey, we evaluated five widely used LLMs: GPT-3.5, GPT-4, Llama-3, BLOOM, and Qwen. We measured how closely these responses aligned with the values of the WEIRD countries and whether they conflicted with human rights principles. To reflect global diversity, we compar…

Multimodal LLMs for Visualization Reconstruction and Understanding
Can Liu, Chunlin Da, Xiaoxiao Long, Yuxiao Yang, Yu Zhang, Yong Wang
https://arxiv.org/abs/2506.21319

Multimodal LLMs for Visualization Reconstruction and Understanding
Visualizations are crucial for data communication, yet understanding them requires comprehension of both visual elements and their underlying data relationships. Current multimodal large models, while effective in natural image understanding, struggle with visualization due to their inability to decode the data-to-visual mapping rules and extract structured information. To address these challenges, we present a novel dataset and train multimodal visualization LLMs specifically designed for unde…

Something provoked me to think about SAM the 1970's news story comprehension and retelling system and the relationship with current LLMs. So I asked Grok, and ok it's on Twitter but it's easily shareable and open, the result is interesting.
https://x.com/Simon_Lucy/status/…

Simon_Lucy (@Simon_Lucy) on X
@grok what was the main achievements that took us from Script Applier Mechanism (SAM) to today's LLMs?

ParEval-Repo: A Benchmark Suite for Evaluating LLMs with Repository-level HPC Translation Tasks
Joshua H. Davis, Daniel Nichols, Ishan Khillan, Abhinav Bhatele
https://arxiv.org/abs/2506.20938

ParEval-Repo: A Benchmark Suite for Evaluating LLMs with Repository-level HPC Translation Tasks
GPGPU architectures have become significantly diverse in recent years, which has led to an emergence of a variety of specialized programming models and software stacks to support them. While portable execution models exist, they still require significant developer effort to port to and optimize for different hardware architectures. Recent advances in large language models (LLMs) can help us reduce some of this programmer burden. In this paper, we present a novel benchmark and testing framework,…

SelfRACG: Enabling LLMs to Self-Express and Retrieve for Code Generation
Qian Dong, Jia Chen, Qingyao Ai, Hongning Wang, Haitao Li, Yi Wu, Yao Hu, Yiqun Liu, Shaoping Ma
https://arxiv.org/abs/2507.19033

SelfRACG: Enabling LLMs to Self-Express and Retrieve for Code Generation
Existing retrieval-augmented code generation (RACG) methods typically use an external retrieval module to fetch semantically similar code snippets used for generating subsequent fragments. However, even for consecutive code fragments, the content often diverges due to logical progression, resulting in a content gap. This gap undermines the performance of current RACG methods, as \textit{external} retrieval modules based on content matching fail to infer the specific information need of LLMs to …

SESR-Eval: Dataset for Evaluating LLMs in the Title-Abstract Screening of Systematic Reviews
Aleksi Huotala, Miikka Kuutila, Mika M\"antyl\"a
https://arxiv.org/abs/2507.19027

SESR-Eval: Dataset for Evaluating LLMs in the Title-Abstract Screening of Systematic Reviews
Background: The use of large language models (LLMs) in the title-abstract screening process of systematic reviews (SRs) has shown promising results, but suffers from limited performance evaluation. Aims: Create a benchmark dataset to evaluate the performance of LLMs in the title-abstract screening process of SRs. Provide evidence whether using LLMs in title-abstract screening in software engineering is advisable. Method: We start with 169 SR research artifacts and find 24 of those to be suitabl…

Reasoning LLMs in the Medical Domain: A Literature Survey
Armin Berger, Sarthak Khanna, David Berghaus, Rafet Sifa
https://arxiv.org/abs/2508.19097 https://

Reasoning LLMs in the Medical Domain: A Literature Survey
The emergence of advanced reasoning capabilities in Large Language Models (LLMs) marks a transformative development in healthcare applications. Beyond merely expanding functional capabilities, these reasoning mechanisms enhance decision transparency and explainability-critical requirements in medical contexts. This survey examines the transformation of medical LLMs from basic information retrieval tools to sophisticated clinical reasoning systems capable of supporting complex healthcare decisio…

Understanding Tool-Integrated Reasoning
Heng Lin, Zhongwen Xu
https://arxiv.org/abs/2508.19201 https://arxiv.org/pdf/2508.19201

Understanding Tool-Integrated Reasoning
We study why Tool-Integrated Reasoning (TIR) makes Large Language Models (LLMs) more capable. While LLMs integrated with tools like Python code interpreters show great promise, a principled theory explaining why this paradigm is effective has been missing. This work provides the first formal proof that TIR fundamentally expands an LLM's capabilities. We demonstrate that tools enable a strict expansion of the model's empirical and feasible support, breaking the capability ceiling of pure-text mo…

A profile of Egune AI, a startup building LLMs for the Mongolian language, as it navigates geopolitics, a lack of resources, and the nascent local tech scene (Viola Zhou/Rest of World)
https://restofworld.org/2025/mongolia-egune-ai-llm/

The Mongolian startup defying Big Tech with its own LLM
Egune is one of several linguistically and culturally aware AI models built by smaller nations to reduce reliance on American and Chinese tech giants.

Large Language Models (LLMs) for Electronic Design Automation (EDA)
Kangwei Xu, Denis Schwachhofer, Jason Blocklove, Ilia Polian, Peter Domanski, Dirk Pfl\"uger, Siddharth Garg, Ramesh Karri, Ozgur Sinanoglu, Johann Knechtel, Zhuorui Zhao, Ulf Schlichtmann, Bing Li
https://arxiv.org/abs/2508.20030…

Large Language Models (LLMs) for Electronic Design Automation (EDA)
With the growing complexity of modern integrated circuits, hardware engineers are required to devote more effort to the full design-to-manufacturing workflow. This workflow involves numerous iterations, making it both labor-intensive and error-prone. Therefore, there is an urgent demand for more efficient Electronic Design Automation (EDA) solutions to accelerate hardware development. Recently, large language models (LLMs) have shown remarkable advancements in contextual comprehension, logical …

Whatever LLMs and gen AI may or may not •actually• be good for, whatever jobs they may or may not actually reshape or displace, right now we’re in the middle of a bubble. The sheer amount of money involved makes it almost impossible to think clearly about this, much less have a useful public discussion. Even well-founded hopes and fears for the tech fuel a fire that I very much do not want to fuel.
8/

It's All About In-Context Learning! Teaching Extremely Low-Resource Languages to LLMs
Yue Li, Zhixue Zhao, Carolina Scarton
https://arxiv.org/abs/2508.19089 https://

It's All About In-Context Learning! Teaching Extremely Low-Resource Languages to LLMs
Extremely low-resource languages, especially those written in rare scripts, as shown in Figure 1, remain largely unsupported by large language models (LLMs). This is due in part to compounding factors such as the lack of training data. This paper delivers the first comprehensive analysis of whether LLMs can acquire such languages purely via in-context learning (ICL), with or without auxiliary alignment signals, and how these methods compare to parameter-efficient fine-tuning (PEFT). We systemat…

Techies are always chasing the mythical tool that will let them "focus on the work" and avoid tedious distractions like "talking to people." AI tools are only the latest to promise this impossible dream.
But talking to people IS the work. You can complain about it on the internet, or take responsibility and make your life a lot easier.
#LLM

Can Designers Avoid Meetings?
LLMs are not the first tool to promise that they can free us from having to talk to people so we can "focus on the work." But talking to people IS the work.

SpecVLM: Enhancing Speculative Decoding of Video LLMs via Verifier-Guided Token Pruning
Yicheng Ji, Jun Zhang, Heming Xia, Jinpeng Chen, Lidan Shou, Gang Chen, Huan Li
https://arxiv.org/abs/2508.16201 …

SpecVLM: Enhancing Speculative Decoding of Video LLMs via Verifier-Guided Token Pruning
Video large language models (Vid-LLMs) have shown strong capabilities in understanding video content. However, their reliance on dense video token representations introduces substantial memory and computational overhead in both prefilling and decoding. To mitigate the information loss of recent video token reduction methods and accelerate the decoding stage of Vid-LLMs losslessly, we introduce SpecVLM, a training-free speculative decoding (SD) framework tailored for Vid-LLMs that incorporates s…

An Investigation on Group Query Hallucination Attacks
Kehao Miao, Xiaolong Jin
https://arxiv.org/abs/2508.19321 https://arxiv.org/pdf/2508.19321

An Investigation on Group Query Hallucination Attacks
With the widespread use of large language models (LLMs), understanding their potential failure modes during user interactions is essential. In practice, users often pose multiple questions in a single conversation with LLMs. Therefore, in this study, we propose Group Query Attack, a technique that simulates this scenario by presenting groups of queries to LLMs simultaneously. We investigate how the accumulated context from consecutive prompts influences the outputs of LLMs. Specifically, we obs…

«the idea of LLMs as a technology that's largely parasitic on status games in a crumbling society does a great deal to explain the recent spate of AI use mandates in businesses. While they're obviously, transparently bad for materially getting work done, they're amazing status boosters, and if people are willing to take on material harm in order to improve their relative status, this behaviour actually makes considerable sense.»
Keeping up appearances | deadSimpleTech
https://deadsimpletech.com/blog/keeping_up_appearances

Thoughts on running local, privacy aware LLMs and other AI/ML via GPT4All, PrivateGPT, OLMo 2 or Ollama?
I have an M3 Macbook Air and limited capacity for faff right now

Our Coding Adventure: Using LLMs to Personalise the Narrative of a Tangible Programming Robot for Preschoolers
Martin Ruskov
https://arxiv.org/abs/2506.20982

Our Coding Adventure: Using LLMs to Personalise the Narrative of a Tangible Programming Robot for Preschoolers
Finding balanced ways to employ Large Language Models (LLMs) in education is a challenge due to inherent risks of poor understanding of the technology and of a susceptible audience. This is particularly so with younger children, who are known to have difficulties with pervasive screen time. Working with a tangible programming robot called Cubetto, we propose an approach to benefit from the capabilities of LLMs by employing such models in the preparation of personalised storytelling, necessary f…

An LLM-powered Natural-to-Robotic Language Translation Framework with Correctness Guarantees
ZhenDong Chen, ZhanShang Nie, ShiXing Wan, JunYi Li, YongTian Cheng, Shuai Zhao
https://arxiv.org/abs/2508.19074

An LLM-powered Natural-to-Robotic Language Translation Framework with Correctness Guarantees
The Large Language Models (LLM) are increasingly being deployed in robotics to generate robot control programs for specific user tasks, enabling embodied intelligence. Existing methods primarily focus on LLM training and prompt design that utilize LLMs to generate executable programs directly from user tasks in natural language. However, due to the inconsistency of the LLMs and the high complexity of the tasks, such best-effort approaches often lead to tremendous programming errors in the gener…

LLMs sind ja auch eher Incompetent-ass-kissing-as-a-service.

LLMs are by definition stick in the past. MCP and other tool use to call the latest version of Go tooling can combat this, e.g. suggesting a refactor to take advantage of new Go features. #GophersUnite

Ah fuck, people are using LLMs for kernel code. They really are going to fuck over everything, aren't they?
https://lwn.net/SubscriberLink/1026558/9f3079fb392c4a9a/

A test of seven AI chatbots' abilities to identify news photos' location, date, and photographer showed all failed to consistently identify photos' provenance (Columbia Journalism Review)
https://www.cjr.org/tow_center/why-ai-models-are-bad-at-verifyi…

Why AI models are bad at verifying photos
Many of the latest large language models, including OpenAI’s recently released GPT-5, tout an impressive capability: reasoning about images. We have seen viral trends of AI models geolocating obscure vacation pictures, impressive demos of LLMs correctly analyzing blurry or cropped photos, and promising results on benchmark tests. These seem to be fueling punditry on whether […]

LEGO-Compiler: Enhancing Neural Compilation Through Translation Composability
Shuoming Zhang, Jiacheng Zhao, Chunwei Xia, Zheng Wang, Yunji Chen, Xiaobing Feng, Huimin Cui
https://arxiv.org/abs/2505.20356

LEGO-Compiler: Enhancing Neural Compilation Through Translation Composability
Large language models (LLMs) have the potential to revolutionize how we design and implement compilers and code translation tools. However, existing LLMs struggle to handle long and complex programs. We introduce LEGO-Compiler, a novel neural compilation system that leverages LLMs to translate high-level languages into assembly code. Our approach centers on three key innovations: LEGO translation, which decomposes the input program into manageable blocks; breaking down the complex compilation p…

The popular meaning of "luddite" is a straw-man. It's a sloppy word with a sloppy meaning now, and it's one we'd do well to watch out for.
The actual reality of who the Luddites were is far more interesting, the center of the hard-fought struggles against owners of factories disrupting entire towns and cities economies with massively terrible results, centralizing power and money and leaving a great number of people without any control of their work, formerly artisans who'd had a hand in their own work, and many automated out of jobs. Luddites destroyed automated looms not because they hated technology. They destroyed automated looms because they were taking the livelihood they depended on, with no recourse, and it was a disaster for a good while, and then millwork has gone from those places probably forever.
The problem now with LLMs and automated research systems is there's very little way for workers and creators to stick their shoes in the machinery. They've tried (https://arxiv.org/abs/2407.12281) but mostly failed, since unlike a factory full of textile workers, the equipment is remote, the automation virtual, an intangible software object that few can access in any meaningful way.

DBMS-LLM Integration Strategies in Industrial and Business Applications: Current Status and Future Challenges
Zhengtong Yan, Gongsheng Yuan, Qingsong Guo, Jiaheng Lu
https://arxiv.org/abs/2507.19254

DBMS-LLM Integration Strategies in Industrial and Business Applications: Current Status and Future Challenges
Modern enterprises are increasingly driven by the DATA+AI paradigm, in which Database Management Systems (DBMSs) and Large Language Models (LLMs) have become two foundational infrastructures powering a wide range of industrial and business applications, such as enterprise analytics, intelligent customer service, and data-driven decision-making. The efficient integration of DBMSs and LLMs within a unified system offers significant opportunities but also introduces new technical challenges. This …

Bias-Adjusted LLM Agents for Human-Like Decision-Making via Behavioral Economics
Ayato Kitadai, Yusuke Fukasawa, Nariaki Nishino
https://arxiv.org/abs/2508.18600 https://…

Bias-Adjusted LLM Agents for Human-Like Decision-Making via Behavioral Economics
Large language models (LLMs) are increasingly used to simulate human decision-making, but their intrinsic biases often diverge from real human behavior--limiting their ability to reflect population-level diversity. We address this challenge with a persona-based approach that leverages individual-level behavioral data from behavioral economics to adjust model biases. Applying this method to the ultimatum game--a standard but difficult benchmark for LLMs--we observe improved alignment between sim…

Spiele seit ein paar Wochen intensiv mit LLMs rum und es ist schon faszinierend, was das so mit der Arbeitsweise/Alltag macht.
Es macht endlich wieder Spaß, Dinge zu recherchieren und Ideen auszuarbeiten. WWW per klassischer Suchmaschine macht schon länger keinen Spaß mehr...
Liegt nicht nur am generierten Schrott, sondern auch an Websites, die so vollgestopft sind, das sie nicht durch die mobile Leitung im Zug passen ;)
LLM filtert und komprimiert :)

When Experimental Economics Meets Large Language Models: Tactics with Evidence
Shu Wang, Zijun Yao, Shuhuai Zhang, Jianuo Gai, Tracy Xiao Liu, Songfa Zhong
https://arxiv.org/abs/2505.21371

When Experimental Economics Meets Large Language Models: Tactics with Evidence
Advancements in large language models (LLMs) have sparked a growing interest in measuring and understanding their behavior through experimental economics. However, there is still a lack of established guidelines for designing economic experiments for LLMs. By combining principles from experimental economics with insights from LLM research in artificial intelligence, we outline and discuss eight practical tactics for conducting experiments with LLMs. We further perform two sets of experiments to…

A highly overlooked perspective, about the current AI discussion, where AI and LLMs are treated as synonyms and so hype and money is funnled into highly inefficient pseudo-solutions when sometimes just a bunch of if-else statements or a state machine would suffice
https://www.youtube.com/watch?v=pnNW4_1DqF

ReCatcher: Towards LLMs Regression Testing for Code Generation
Altaf Allah Abbassi, Leuson Da Silva, Amin Nikanjam, Foutse Khomh
https://arxiv.org/abs/2507.19390 https://…

ReCatcher: Towards LLMs Regression Testing for Code Generation
Large Language Models (LLMs) for code generation evolve rapidly through fine-tuning, merging, or new model releases. However, such updates can introduce regressions, not only in correctness but also in code quality and performance. To address this, we present ReCatcher, a regression testing framework for Python code generation. ReCatcher systematically compares two LLMs, typically a current model and a candidate update, across three dimensions: logical correctness, static code quality, and exec…

Etwas mehr der heute besonders häufig geteilten #News:
Grammatikfehler machen Prompt Injections wahrscheinlicher
https:…

Can Structured Templates Facilitate LLMs in Tackling Harder Tasks? : An Exploration of Scaling Laws by Difficulty
Zhichao Yang, Zhaoxin Fan, Gen Li, Yuanze Hu, Xinyu Wang, Ye Qiu, Xin Wang, Yifan Sun, Wenjun Wu
https://arxiv.org/abs/2508.19069

Can Structured Templates Facilitate LLMs in Tackling Harder Tasks? : An Exploration of Scaling Laws by Difficulty
Structured, procedural reasoning is essential for Large Language Models (LLMs), especially in mathematics. While post-training methods have improved LLM performance, they still fall short in capturing deep procedural logic on complex tasks. To tackle the issue, in this paper, we first investigate this limitation and uncover a novel finding: a Scaling Law by Difficulty, which reveals that model performance follows a U-shaped curve with respect to training data complexity -- excessive low-difficu…

Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning
Alan Li, Yixin Liu, Arpan Sarkar, Doug Downey, Arman Cohan
https://arxiv.org/abs/2508.19202 ht…

Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning
Scientific problem solving poses unique challenges for LLMs, requiring both deep domain knowledge and the ability to apply such knowledge through complex reasoning. While automated scientific reasoners hold great promise for assisting human scientists, there is currently no widely adopted holistic benchmark for evaluating scientific reasoning, and few approaches systematically disentangle the distinct roles of knowledge and reasoning in these tasks. To address these gaps, we introduce SciReas, …

Injecting External Knowledge into the Reasoning Process Enhances Retrieval-Augmented Generation
Minghao Tang, Shiyu Ni, Jiafeng Guo, Keping Bi
https://arxiv.org/abs/2507.19333 h…

Injecting External Knowledge into the Reasoning Process Enhances Retrieval-Augmented Generation
Retrieval-augmented generation (RAG) has been widely adopted to augment large language models (LLMs) with external knowledge for knowledge-intensive tasks. However, its effectiveness is often undermined by the presence of noisy (i.e., low-quality) retrieved passages. Enhancing LLMs' robustness to such noise is critical for improving the reliability of RAG systems. Recent advances have equipped LLMs with strong reasoning and self-reflection capabilities, allowing them to identify and correct err…

APT-LLM: Exploiting Arbitrary-Precision Tensor Core Computing for LLM Acceleration
Shaobo Ma, Chao Fang, Haikuo Shao, Zhongfeng Wang
https://arxiv.org/abs/2508.19087 https://

APT-LLM: Exploiting Arbitrary-Precision Tensor Core Computing for LLM Acceleration
Large language models (LLMs) have revolutionized AI applications, yet their enormous computational demands severely limit deployment and real-time performance. Quantization methods can help reduce computational costs, however, attaining the extreme efficiency associated with ultra-low-bit quantized LLMs at arbitrary precision presents challenges on GPUs. This is primarily due to the limited support for GPU Tensor Cores, inefficient memory management, and inflexible kernel optimizations. To tack…

Whatever LLMs and gen AI may or may not •actually• be good for, whatever jobs they may or may not actually reshape or displace, right now we’re in the middle of a bubble. The sheer amount of money involved makes it almost impossible to think clearly about this, much less have a useful public discussion. Even well-founded hopes and fears for the tech fuel a fire that I very much do not want to fuel.
8/

The term "context engineering" is gaining traction over "prompt engineering" as it better describes the skill of providing LLMs with the necessary information (Simon Willison/Simon Willison's Weblog)
https://simonwillison.net/2025/Jun/27/context-…

Stand on The Shoulders of Giants: Building JailExpert from Previous Attack Experience
Xi Wang, Songlei Jian, Shasha Li, Xiaopeng Li, Bin Ji, Jun Ma, Xiaodong Liu, Jing Wang, Feilong Bao, Jianfeng Zhang, Baosheng Wang, Jie Yu
https://arxiv.org/abs/2508.19292

Stand on The Shoulders of Giants: Building JailExpert from Previous Attack Experience
Large language models (LLMs) generate human-aligned content under certain safety constraints. However, the current known technique ``jailbreak prompt'' can circumvent safety-aligned measures and induce LLMs to output malicious content. Research on Jailbreaking can help identify vulnerabilities in LLMs and guide the development of robust security frameworks. To circumvent the issue of attack templates becoming obsolete as models evolve, existing methods adopt iterative mutation and dynamic optim…

«the idea of LLMs as a technology that's largely parasitic on status games in a crumbling society does a great deal to explain the recent spate of AI use mandates in businesses. While they're obviously, transparently bad for materially getting work done, they're amazing status boosters, and if people are willing to take on material harm in order to improve their relative status, this behaviour actually makes considerable sense.»
Keeping up appearances | deadSimpleTech
https://deadsimpletech.com/blog/keeping_up_appearances

ReST-RL: Achieving Accurate Code Reasoning of LLMs with Optimized Self-Training and Decoding
Sining Zhoubian, Dan Zhang, Yuxiao Dong, Jie Tang
https://arxiv.org/abs/2508.19576 h…

ReST-RL: Achieving Accurate Code Reasoning of LLMs with Optimized Self-Training and Decoding
With respect to improving the reasoning accuracy of LLMs, the representative reinforcement learning (RL) method GRPO faces failure due to insignificant reward variance, while verification methods based on process reward models (PRMs) suffer from difficulties with training data acquisition and verification effectiveness. To tackle these problems, this paper introduces ReST-RL, a unified LLM RL paradigm that significantly improves LLM's code reasoning ability by combining an improved GRPO algorit…

BLOCKS: Blockchain-supported Cross-Silo Knowledge Sharing for Efficient LLM Services
Zhaojiacheng Zhou, Hongze Liu, Shijing Yuan, Hanning Zhang, Jiong Lou, Chentao Wu, Jie Li
https://arxiv.org/abs/2506.21033

BLOCKS: Blockchain-supported Cross-Silo Knowledge Sharing for Efficient LLM Services
The hallucination problem of Large Language Models (LLMs) has increasingly drawn attention. Augmenting LLMs with external knowledge is a promising solution to address this issue. However, due to privacy and security concerns, a vast amount of downstream task-related knowledge remains dispersed and isolated across various "silos," making it difficult to access. To bridge this knowledge gap, we propose a blockchain-based external knowledge framework that coordinates multiple knowledge silos to pr…

By default, common LLMs don't use current code features, and even higher-power ones are a coin flip. Go analysis tools, and the associated MCP server, helps push back against this so emitted code gets modernized on the way out of the model. #GophersUnite

Filtering for Creativity: Adaptive Prompting for Multilingual Riddle Generation in LLMs
Duy Le, Kent Ziti, Evan Girard-Sun, Sean O'Brien, Vasu Sharma, Kevin Zhu
https://arxiv.org/abs/2508.18709

Filtering for Creativity: Adaptive Prompting for Multilingual Riddle Generation in LLMs
Multilingual riddle generation challenges large language models (LLMs) to balance cultural fluency with creative abstraction. Standard prompting strategies -- zero-shot, few-shot, chain-of-thought -- tend to reuse memorized riddles or perform shallow paraphrasing. We introduce Adaptive Originality Filtering (AOF), a prompting framework that filters redundant generations using cosine-based similarity rejection, while enforcing lexical novelty and cross-lingual fidelity. Evaluated across three LL…

Leveraging LLMs for Automated Translation of Legacy Code: A Case Study on PL/SQL to Java Transformation
Lola Solovyeva, Eduardo Carneiro Oliveira, Shiyu Fan, Alper Tuncay, Shamil Gareev, Andrea Capiluppi
https://arxiv.org/abs/2508.19663

Leveraging LLMs for Automated Translation of Legacy Code: A Case Study on PL/SQL to Java Transformation
The VT legacy system, comprising approximately 2.5 million lines of PL/SQL code, lacks consistent documentation and automated tests, posing significant challenges for refactoring and modernisation. This study investigates the feasibility of leveraging large language models (LLMs) to assist in translating PL/SQL code into Java for the modernised "VTF3" system. By leveraging a dataset comprising 10 PL/SQL-to-Java code pairs and 15 Java classes, which collectively established a domain model for th…

2025 fühlt sich echt spät an für einen Informatiker, sich mit LLMs/AI zu beschäftigen, aber irgendwie war ich 2009/12 schon dabei, damals an der Uni
Ist ganz gut, die "Vorgeschichte" zu kennen und dann den ersten Hype-Buckel abzuwarten ;) Nicht geplant, aber gut gelaufen für mich, habe ich das Gefühl.

Integrating Time Series into LLMs via Multi-layer Steerable Embedding Fusion for Enhanced Forecasting
Zhuomin Chen, Dan Li, Jiahui Zhou, Shunyu Wu, Haozheng Ye, Jian Lou, See-Kiong Ng
https://arxiv.org/abs/2508.16059

Integrating Time Series into LLMs via Multi-layer Steerable Embedding Fusion for Enhanced Forecasting
Time series (TS) data are ubiquitous across various application areas, rendering time series forecasting (TSF) a fundamental task. With the astounding advances in large language models (LLMs), a variety of methods have been developed to adapt LLMs for time series forecasting. Despite unlocking the potential of LLMs in comprehending TS data, existing methods are inherently constrained by their shallow integration of TS information, wherein LLMs typically access TS representations at shallow laye…

IsaMini: Redesigned Isabelle Proof Lanugage for Machine Learning
Qiyuan Xu, Renxi Wang, Haonan Li, David Sanan, Conrad Watt
https://arxiv.org/abs/2507.18885 https://

IsaMini: Redesigned Isabelle Proof Lanugage for Machine Learning
Neural Theorem Proving (NTP) employs deep learning methods, particularly Large Language Models (LLMs), to automate formal proofs in proof assistants. This approach holds promise for reducing the dramatic labor costs or computation costs required in proof engineering, which is fundamental to formal verification and other software engineering methods. The paper explores the potential of improving NTP by redesigning the proof language, given that LLMs' capabilities depend highly on representations…

Boardwalk: Towards a Framework for Creating Board Games with LLMs
\'Alvaro Guglielmin Becker, Gabriel Bauer de Oliveira, Lana Bertoldo Rossato, Anderson Rocha Tavares
https://arxiv.org/abs/2508.16447

Boardwalk: Towards a Framework for Creating Board Games with LLMs
Implementing board games in code can be a time-consuming task. However, Large Language Models (LLMs) have been proven effective at generating code for domain-specific tasks with simple contextual information. We aim to investigate whether LLMs can implement digital versions of board games from rules described in natural language. This would be a step towards an LLM-assisted framework for quick board game code generation. We expect to determine the main challenges for LLMs to implement the board…

ConfTuner: Training Large Language Models to Express Their Confidence Verbally
Yibo Li, Miao Xiong, Jiaying Wu, Bryan Hooi
https://arxiv.org/abs/2508.18847 https://

ConfTuner: Training Large Language Models to Express Their Confidence Verbally
Large Language Models (LLMs) are increasingly deployed in high-stakes domains such as science, law, and healthcare, where accurate expressions of uncertainty are essential for reliability and trust. However, current LLMs are often observed to generate incorrect answers with high confidence, a phenomenon known as "overconfidence". Recent efforts have focused on calibrating LLMs' verbalized confidence: i.e., their expressions of confidence in text form, such as "I am 80% confident that...". Exist…

Exploring the Use of LLMs for Requirements Specification in an IT Consulting Company
Liliana Pasquale, Azzurra Ragone, Emanuele Piemontese, Armin Amiri Darban
https://arxiv.org/abs/2507.19113

Exploring the Use of LLMs for Requirements Specification in an IT Consulting Company
In practice, requirements specification remains a critical challenge. The knowledge necessary to generate a specification can often be fragmented across diverse sources (e.g., meeting minutes, emails, and high-level product descriptions), making the process cumbersome and time-consuming. In this paper, we report our experience using large language models (LLMs) in an IT consulting company to automate the requirements specification process. In this company, requirements are specified using a Fun…

RL-Finetuned LLMs for Privacy-Preserving Synthetic Rewriting
Zhan Shi, Yefeng Yuan, Yuhong Liu, Liang Cheng, Yi Fang
https://arxiv.org/abs/2508.19286 https://

RL-Finetuned LLMs for Privacy-Preserving Synthetic Rewriting
The performance of modern machine learning systems depends on access to large, high-quality datasets, often sourced from user-generated content or proprietary, domain-specific corpora. However, these rich datasets inherently contain sensitive personal information, raising significant concerns about privacy, data security, and compliance with regulatory frameworks. While conventional anonymization techniques can remove explicit identifiers, such removal may result in performance drop in downstre…

Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
Haoang Chi, He Li, Wenjing Yang, Feng Liu, Long Lan, Xiaoguang Ren, Tongliang Liu, Bo Han
https://arxiv.org/abs/2506.21215

Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
Causal reasoning capability is critical in advancing large language models (LLMs) toward strong artificial intelligence. While versatile LLMs appear to have demonstrated capabilities in understanding contextual causality and providing responses that obey the laws of causality, it remains unclear whether they perform genuine causal reasoning akin to humans. However, current evidence indicates the contrary. Specifically, LLMs are only capable of performing shallow (level-1) causal reasoning, prim…

Distilling a Small Utility-Based Passage Selector to Enhance Retrieval-Augmented Generation
Hengran Zhang, Keping Bi, Jiafeng Guo, Jiaming Zhang, Shuaiqiang Wang, Dawei Yin, Xueqi Cheng
https://arxiv.org/abs/2507.19102

Distilling a Small Utility-Based Passage Selector to Enhance Retrieval-Augmented Generation
Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating retrieved information. Standard retrieval process prioritized relevance, focusing on topical alignment between queries and passages. In contrast, in RAG, the emphasis has shifted to utility, which considers the usefulness of passages for generating accurate answers. Despite empirical evidence showing the benefits of utility-based retrieval in RAG, the high computational cost of using LLMs for utility jud…

KI-Update kompakt: LLMs für Malware, Anti-Human-Bias, Sutton, Chatbots
Das "KI-Update" liefert werktäglich eine Zusammenfassung der wichtigsten KI-Entwicklungen.
https://www.

Interleaving Large Language Models for Compiler Testing
Yunbo Ni, Shaohua Li
https://arxiv.org/abs/2508.18955 https://arxiv.org/pdf/2508.18955

Interleaving Large Language Models for Compiler Testing
Testing compilers with AI models, especially large language models (LLMs), has shown great promise. However, current approaches struggle with two key problems: The generated programs for testing compilers are often too simple, and extensive testing with the LLMs is computationally expensive. In this paper, we propose a novel compiler testing framework that decouples the testing process into two distinct phases: an offline phase and an online phase. In the offline phase, we use LLMs to generate …

FALCON: Autonomous Cyber Threat Intelligence Mining with LLMs for IDS Rule Generation
Shaswata Mitra, Azim Bazarov, Martin Duclos, Sudip Mittal, Aritran Piplai, Md Rayhanur Rahman, Edward Zieglar, Shahram Rahimi
https://arxiv.org/abs/2508.18684

FALCON: Autonomous Cyber Threat Intelligence Mining with LLMs for IDS Rule Generation
Signature-based Intrusion Detection Systems (IDS) detect malicious activities by matching network or host activity against predefined rules. These rules are derived from extensive Cyber Threat Intelligence (CTI), which includes attack signatures and behavioral patterns obtained through automated tools and manual threat analysis, such as sandboxing. The CTI is then transformed into actionable rules for the IDS engine, enabling real-time detection and prevention. However, the constant evolution o…

From BERT to LLMs: Comparing and Understanding Chinese Classifier Prediction in Language Models
ZiqiZhang, Jianfei Ma, Emmanuele Chersoni, Jieshun You, Zhaoxin Feng
https://arxiv.org/abs/2508.18253

From BERT to LLMs: Comparing and Understanding Chinese Classifier Prediction in Language Models
Classifiers are an important and defining feature of the Chinese language, and their correct prediction is key to numerous educational applications. Yet, whether the most popular Large Language Models (LLMs) possess proper knowledge the Chinese classifiers is an issue that has largely remain unexplored in the Natural Language Processing (NLP) literature. To address such a question, we employ various masking strategies to evaluate the LLMs' intrinsic ability, the contribution of different sent…

A Systematic Approach to Predict the Impact of Cybersecurity Vulnerabilities Using LLMs
Anders M{\o}lmen H{\o}st, Pierre Lison, Leon Moonen
https://arxiv.org/abs/2508.18439 http…

A Systematic Approach to Predict the Impact of Cybersecurity Vulnerabilities Using LLMs
Vulnerability databases, such as the National Vulnerability Database (NVD), offer detailed descriptions of Common Vulnerabilities and Exposures (CVEs), but often lack information on their real-world impact, such as the tactics, techniques, and procedures (TTPs) that adversaries may use to exploit the vulnerability. However, manually linking CVEs to their corresponding TTPs is a challenging and time-consuming task, and the high volume of new vulnerabilities published annually makes automated sup…

InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning Styles
Zizhen Li, Chuanhao Li, Yibin Wang, Qi Chen, Diping Song, Yukang Feng, Jianwen Sun, Jiaxin Ai, Fanrui Zhang, Mingzhu Sun, Kaipeng Zhang
https://arxiv.org/abs/2508.16072

InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning Styles
LLMs have shown strong performance on human-centric reasoning tasks. While previous evaluations have explored whether LLMs can infer intentions or detect deception, they often overlook the individualized reasoning styles that influence how people interpret and act in social contexts. Social deduction games (SDGs) provide a natural testbed for evaluating individualized reasoning styles, where different players may adopt diverse but contextually valid reasoning strategies under identical conditio…

CycleDistill: Bootstrapping Machine Translation using LLMs with Cyclical Distillation
Deepon Halder, Thanmay Jayakumar, Raj Dabre
https://arxiv.org/abs/2506.19952

CycleDistill: Bootstrapping Machine Translation using LLMs with Cyclical Distillation
Large language models (LLMs), despite their ability to perform few-shot machine translation (MT), often lag behind dedicated MT systems trained on parallel corpora, which are crucial for high quality machine translation (MT). However, parallel corpora are often scarce or non-existent for low-resource languages. In this paper, we propose CycleDistill, a bootstrapping approach leveraging LLMs and few-shot translation to obtain high-quality MT systems. CycleDistill involves iteratively generating …

Integrating Time Series into LLMs via Multi-layer Steerable Embedding Fusion for Enhanced Forecasting
Zhuomin Chen, Dan Li, Jiahui Zhou, Shunyu Wu, Haozheng Ye, Jian Lou, See-Kiong Ng
https://arxiv.org/abs/2508.16059

Integrating Time Series into LLMs via Multi-layer Steerable Embedding Fusion for Enhanced Forecasting
Time series (TS) data are ubiquitous across various application areas, rendering time series forecasting (TSF) a fundamental task. With the astounding advances in large language models (LLMs), a variety of methods have been developed to adapt LLMs for time series forecasting. Despite unlocking the potential of LLMs in comprehending TS data, existing methods are inherently constrained by their shallow integration of TS information, wherein LLMs typically access TS representations at shallow laye…

MemoCoder: Automated Function Synthesis using LLM-Supported Agents
Yiping Jia, Zhen Ming Jiang, Shayan Noei, Ying Zou
https://arxiv.org/abs/2507.18812 https://

MemoCoder: Automated Function Synthesis using LLM-Supported Agents
With the widespread adoption of Large Language Models (LLMs) such as GitHub Copilot and ChatGPT, developers increasingly rely on AI-assisted tools to support code generation. While LLMs can generate syntactically correct solutions for well-structured programming tasks, they often struggle with challenges that require iterative debugging, error handling, or adaptation to diverse problem structures. Existing approaches such as fine-tuning or self-repair strategies either require costly retraining…

Thinking Before You Speak: A Proactive Test-time Scaling Approach
Cong Li, Wenchang Chai, Hejun Wu, Yan Pan, Pengxu Wei, Liang Lin
https://arxiv.org/abs/2508.18648 https://

Thinking Before You Speak: A Proactive Test-time Scaling Approach
Large Language Models (LLMs) often exhibit deficiencies with complex reasoning tasks, such as maths, which we attribute to the discrepancy between human reasoning patterns and those presented in the LLMs' training data. When dealing with complex problems, humans tend to think carefully before expressing solutions. However, they often do not articulate their inner thoughts, including their intentions and chosen methodologies. Consequently, critical insights essential for bridging reasoning steps…

SoK: Large Language Model Copyright Auditing via Fingerprinting
Shuo Shao, Yiming Li, Yu He, Hongwei Yao, Wenyuan Yang, Dacheng Tao, Zhan Qin
https://arxiv.org/abs/2508.19843 ht…

SoK: Large Language Model Copyright Auditing via Fingerprinting
The broad capabilities and substantial resources required to train Large Language Models (LLMs) make them valuable intellectual property, yet they remain vulnerable to copyright infringement, such as unauthorized use and model theft. LLM fingerprinting, a non-intrusive technique that extracts and compares the distinctive features from LLMs to identify infringements, offers a promising solution to copyright auditing. However, its reliability remains uncertain due to the prevalence of diverse mod…

Tracking World States with Language Models: State-Based Evaluation Using Chess
Romain Harang, Jason Naradowsky, Yaswitha Gujju, Yusuke Miyao
https://arxiv.org/abs/2508.19851 htt…

Tracking World States with Language Models: State-Based Evaluation Using Chess
Large Language Models (LLMs) exhibit emergent capabilities in structured domains, suggesting they may implicitly internalize high-fidelity representations of world models. While probing techniques have shown promising signs of this in scientific and game-based settings, they rely on model-specific internal activations, which limit interpretability and generalizability. In this work, we propose a model-agnostic, state-based evaluation framework using chess as a benchmark to assess whether LLMs p…

Generative Interfaces for Language Models
Jiaqi Chen, Yanzhe Zhang, Yutong Zhang, Yijia Shao, Diyi Yang
https://arxiv.org/abs/2508.19227 https://arxiv.org/…

Generative Interfaces for Language Models
Large language models (LLMs) are increasingly seen as assistants, copilots, and consultants, capable of supporting a wide range of tasks through natural conversation. However, most systems remain constrained by a linear request-response format that often makes interactions inefficient in multi-turn, information-dense, and exploratory tasks. To address these limitations, we propose Generative Interfaces for Language Models, a paradigm in which LLMs respond to user queries by proactively generati…

Training Language Model Agents to Find Vulnerabilities with CTF-Dojo
Terry Yue Zhuo, Dingmin Wang, Hantian Ding, Varun Kumar, Zijian Wang
https://arxiv.org/abs/2508.18370 https:…

Training Language Model Agents to Find Vulnerabilities with CTF-Dojo
Large language models (LLMs) have demonstrated exceptional capabilities when trained within executable runtime environments, notably excelling at software engineering tasks through verified feedback loops. Yet, scalable and generalizable execution-grounded environments remain scarce, limiting progress in training more capable ML agents. We introduce CTF-Dojo, the first large-scale executable runtime tailored for training LLMs with verifiable feedback, featuring 658 fully functional Capture-The-…

Disabling Self-Correction in Retrieval-Augmented Generation via Stealthy Retriever Poisoning
Yanbo Dai, Zhenlan Ji, Zongjie Li, Kuan Li, Shuai Wang
https://arxiv.org/abs/2508.20083

Disabling Self-Correction in Retrieval-Augmented Generation via Stealthy Retriever Poisoning
Retrieval-Augmented Generation (RAG) has become a standard approach for improving the reliability of large language models (LLMs). Prior work demonstrates the vulnerability of RAG systems by misleading them into generating attacker-chosen outputs through poisoning the knowledge base. However, this paper uncovers that such attacks could be mitigated by the strong \textit{self-correction ability (SCA)} of modern LLMs, which can reject false context once properly configured. This SCA poses a signi…

Sense of Self and Time in Borderline Personality. A Comparative Robustness Study with Generative AI
Marcin Moskalewicz, Anna Sterna, Marek Pokropski, Paula Flores
https://arxiv.org/abs/2508.19008

Sense of Self and Time in Borderline Personality. A Comparative Robustness Study with Generative AI
This study examines the capacity of large language models (LLMs) to support phenomenological qualitative analysis of first-person experience in Borderline Personality Disorder (BPD), understood as a disorder of temporality and selfhood. Building on a prior human-led thematic analysis of 24 inpatients' life-story interviews, we compared three LLMs (OpenAI GPT-4o, Google Gemini 2.5 Pro, Anthropic Claude Opus 4) prompted to mimic the interpretative style of the original investigators. The models w…

GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
Lakshya A Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, Omar Khattab
https://arx…

GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
Large language models (LLMs) are increasingly adapted to downstream tasks via reinforcement learning (RL) methods like Group Relative Policy Optimization (GRPO), which often require thousands of rollouts to learn new tasks. We argue that the interpretable nature of language can often provide a much richer learning medium for LLMs, compared with policy gradients derived from sparse, scalar rewards. To test this, we introduce GEPA (Genetic-Pareto), a prompt optimizer that thoroughly incorporates …

Requirements Development and Formalization for Reliable Code Generation: A Multi-Agent Vision
Xu Lu, Weisong Sun, Yiran Zhang, Ming Hu, Cong Tian, Zhi Jin, Yang Liu
https://arxiv.org/abs/2508.18675

Requirements Development and Formalization for Reliable Code Generation: A Multi-Agent Vision
Automated code generation has long been considered the holy grail of software engineering. The emergence of Large Language Models (LLMs) has catalyzed a revolutionary breakthrough in this area. However, existing methods that only rely on LLMs remain inadequate in the quality of generated code, offering no guarantees of satisfying practical requirements. They lack a systematic strategy for requirements development and modeling. Recently, LLM-based agents typically possess powerful abilities and …

Interactive Evaluation of Large Language Models for Multi-Requirement Software Engineering Tasks
Dimitrios Rontogiannis, Maxime Peyrard, Nicolas Baldwin, Martin Josifoski, Robert West, Dimitrios Gunopulos
https://arxiv.org/abs/2508.18905

Interactive Evaluation of Large Language Models for Multi-Requirement Software Engineering Tasks
Standard single-turn, static benchmarks fall short in evaluating the nuanced capabilities of Large Language Models (LLMs) on complex tasks such as software engineering. In this work, we propose a novel interactive evaluation framework that assesses LLMs on multi-requirement programming tasks through structured, feedback-driven dialogue. Each task is modeled as a requirement dependency graph, and an ``interviewer'' LLM, aware of the ground-truth solution, provides minimal, targeted hints to an `…

Breaking the Trade-Off Between Faithfulness and Expressiveness for Large Language Models
Chenxu Yang, Qingyi Si, Zheng Lin
https://arxiv.org/abs/2508.18651 https://

Breaking the Trade-Off Between Faithfulness and Expressiveness for Large Language Models
Grounding responses in external knowledge represents an effective strategy for mitigating hallucinations in Large Language Models (LLMs). However, current LLMs struggle to seamlessly integrate knowledge while simultaneously maintaining faithfulness (or fidelity) and expressiveness, capabilities that humans naturally possess. This limitation results in outputs that either lack support from external knowledge, thereby compromising faithfulness, or appear overly verbose and unnatural, thus sacrifi…

Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs
Yu Yan, Sheng Sun, Zhe Wang, Yijun Lin, Zenghao Duan, zhifei zheng, Min Liu, Zhiyi yin, Jianping Zhang
https://arxiv.org/abs/2508.16347

Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs
With the development of Large Language Models (LLMs), numerous efforts have revealed their vulnerabilities to jailbreak attacks. Although these studies have driven the progress in LLMs' safety alignment, it remains unclear whether LLMs have internalized authentic knowledge to deal with real-world crimes, or are merely forced to simulate toxic language patterns. This ambiguity raises concerns that jailbreak success is often attributable to a hallucination loop between jailbroken LLM and judger L…

Identifying Fine-grained Forms of Populism in Political Discourse: A Case Study on Donald Trump's Presidential Campaigns
Ilias Chalkidis, Stephanie Brandl, Paris Aslanidis
https://arxiv.org/abs/2507.19303

Identifying Fine-grained Forms of Populism in Political Discourse: A Case Study on Donald Trump's Presidential Campaigns
Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide range of instruction-following tasks, yet their grasp of nuanced social science concepts remains underexplored. This paper examines whether LLMs can identify and classify fine-grained forms of populism, a complex and contested concept in both academic and media debates. To this end, we curate and release novel datasets specifically designed to capture populist discourse. We evaluate a range of pre-trained (larg…

Can LLMs Replace Humans During Code Chunking?
Christopher Glasz, Emily Escamilla, Eric O. Scott, Anand Patel, Jacob Zimmer, Colin Diggs, Michael Doyle, Scott Rosen, Nitin Naik, Justin F. Brunelle, Samruddhi Thaker, Parthav Poudel, Arun Sridharan, Amit Madan, Doug Wendt, William Macke, Thomas Schill
https://arxiv.org/abs/2506.198…

Can LLMs Replace Humans During Code Chunking?
Large language models (LLMs) have become essential tools in computer science, especially for tasks involving code understanding and generation. However, existing work does not address many of the unique challenges presented by code written for government applications. In particular, government enterprise software is often written in legacy languages like MUMPS or assembly language code (ALC) and the overall token lengths of these systems exceed the context window size for current commercially a…

Structuralist Approach to AI Literary Criticism: Leveraging Greimas Semiotic Square for Large Language Models
Fangzhou Dong, Yifan Zeng, Yingpeng Sang, Hong Shen
https://arxiv.org/abs/2506.21360

Structuralist Approach to AI Literary Criticism: Leveraging Greimas Semiotic Square for Large Language Models
Large Language Models (LLMs) excel in understanding and generating text but struggle with providing professional literary criticism for works with profound thoughts and complex narratives. This paper proposes GLASS (Greimas Literary Analysis via Semiotic Square), a structured analytical framework based on Greimas Semiotic Square (GSS), to enhance LLMs' ability to conduct in-depth literary analysis. GLASS facilitates the rapid dissection of narrative structures and deep meanings in narrative wor…

JsDeObsBench: Measuring and Benchmarking LLMs for JavaScript Deobfuscation
Guoqiang Chen, Xin Jin, Zhiqiang Lin
https://arxiv.org/abs/2506.20170 https://…

JsDeObsBench: Measuring and Benchmarking LLMs for JavaScript Deobfuscation
Deobfuscating JavaScript (JS) code poses a significant challenge in web security, particularly as obfuscation techniques are frequently used to conceal malicious activities within scripts. While Large Language Models (LLMs) have recently shown promise in automating the deobfuscation process, transforming detection and mitigation strategies against these obfuscated threats, a systematic benchmark to quantify their effectiveness and limitations has been notably absent. To address this gap, we pre…

Harnessing Rule-Based Reinforcement Learning for Enhanced Grammatical Error Correction
Yilin Li, Xunjian Yin, Yilin Chen, Xiaojun Wan
https://arxiv.org/abs/2508.18780 https://…

Harnessing Rule-Based Reinforcement Learning for Enhanced Grammatical Error Correction
Grammatical error correction is a significant task in NLP. Traditional methods based on encoder-decoder models have achieved certain success, but the application of LLMs in this field is still underexplored. Current research predominantly relies on supervised fine-tuning to train LLMs to directly generate the corrected sentence, which limits the model's powerful reasoning ability. To address this limitation, we propose a novel framework based on Rule-Based RL. Through experiments on the Chinese…

"What's Up, Doc?": Analyzing How Users Seek Health Information in Large-Scale Conversational AI Datasets
Akshay Paruchuri, Maryam Aziz, Rohit Vartak, Ayman Ali, Best Uchehara, Xin Liu, Ishan Chatterjee, Monica Agrawal
https://arxiv.org/abs/2506.21532

"What's Up, Doc?": Analyzing How Users Seek Health Information in Large-Scale Conversational AI Datasets
People are increasingly seeking healthcare information from large language models (LLMs) via interactive chatbots, yet the nature and inherent risks of these conversations remain largely unexplored. In this paper, we filter large-scale conversational AI datasets to achieve HealthChat-11K, a curated dataset of 11K real-world conversations composed of 25K user messages. We use HealthChat-11K and a clinician-driven taxonomy for how users interact with LLMs when seeking healthcare information in or…

InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning Styles
Zizhen Li, Chuanhao Li, Yibin Wang, Qi Chen, Diping Song, Yukang Feng, Jianwen Sun, Jiaxin Ai, Fanrui Zhang, Mingzhu Sun, Kaipeng Zhang
https://arxiv.org/abs/2508.16072

InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning Styles
LLMs have shown strong performance on human-centric reasoning tasks. While previous evaluations have explored whether LLMs can infer intentions or detect deception, they often overlook the individualized reasoning styles that influence how people interpret and act in social contexts. Social deduction games (SDGs) provide a natural testbed for evaluating individualized reasoning styles, where different players may adopt diverse but contextually valid reasoning strategies under identical conditio…

Retrieval-Confused Generation is a Good Defender for Privacy Violation Attack of Large Language Models
Wanli Peng, Xin Chen, Hang Fu, XinYu He, Xue Yiming, Juan Wen
https://arxiv.org/abs/2506.19889

Retrieval-Confused Generation is a Good Defender for Privacy Violation Attack of Large Language Models
Recent advances in large language models (LLMs) have made a profound impact on our society and also raised new security concerns. Particularly, due to the remarkable inference ability of LLMs, the privacy violation attack (PVA), revealed by Staab et al., introduces serious personal privacy issues. Existing defense methods mainly leverage LLMs to anonymize the input query, which requires costly inference time and cannot gain satisfactory defense performance. Moreover, directly rejecting the PVA …

Domain Knowledge-Enhanced LLMs for Fraud and Concept Drift Detection
Ali \c{S}enol, Garima Agrawal, Huan Liu
https://arxiv.org/abs/2506.21443 https://arxiv.org/pdf/2506.21443 https://arxiv.org/html/2506.21443
arXiv:2506.21443v1 Announce Type: new
Abstract: Detecting deceptive conversations on dynamic platforms is increasingly difficult due to evolving language patterns and Concept Drift (CD)\-i.e., semantic or topical shifts that alter the context or intent of interactions over time. These shifts can obscure malicious intent or mimic normal dialogue, making accurate classification challenging. While Large Language Models (LLMs) show strong performance in natural language tasks, they often struggle with contextual ambiguity and hallucinations in risk\-sensitive scenarios. To address these challenges, we present a Domain Knowledge (DK)\-Enhanced LLM framework that integrates pretrained LLMs with structured, task\-specific insights to perform fraud and concept drift detection. The proposed architecture consists of three main components: (1) a DK\-LLM module to detect fake or deceptive conversations; (2) a drift detection unit (OCDD) to determine whether a semantic shift has occurred; and (3) a second DK\-LLM module to classify the drift as either benign or fraudulent. We first validate the value of domain knowledge using a fake review dataset and then apply our full framework to SEConvo, a multiturn dialogue dataset that includes various types of fraud and spam attacks. Results show that our system detects fake conversations with high accuracy and effectively classifies the nature of drift. Guided by structured prompts, the LLaMA\-based implementation achieves 98\% classification accuracy. Comparative studies against zero\-shot baselines demonstrate that incorporating domain knowledge and drift awareness significantly improves performance, interpretability, and robustness in high\-stakes NLP applications.
toXiv_bot_toot

Debiasing Multilingual LLMs in Cross-lingual Latent Space
Qiwei Peng, Guimin Hu, Yekun Chai, Anders S{\o}gaard
https://arxiv.org/abs/2508.17948 https://arx…

Debiasing Multilingual LLMs in Cross-lingual Latent Space
Debiasing techniques such as SentDebias aim to reduce bias in large language models (LLMs). Previous studies have evaluated their cross-lingual transferability by directly applying these methods to LLM representations, revealing their limited effectiveness across languages. In this work, we therefore propose to perform debiasing in a joint latent space rather than directly on LLM representations. We construct a well-aligned cross-lingual latent space using an autoencoder trained on parallel TED…

Agri-Query: A Case Study on RAG vs. Long-Context LLMs for Cross-Lingual Technical Question Answering
Julius Gun, Timo Oksanen
https://arxiv.org/abs/2508.18093 https://

Agri-Query: A Case Study on RAG vs. Long-Context LLMs for Cross-Lingual Technical Question Answering
We present a case study evaluating large language models (LLMs) with 128K-token context windows on a technical question answering (QA) task. Our benchmark is built on a user manual for an agricultural machine, available in English, French, and German. It simulates a cross-lingual information retrieval scenario where questions are posed in English against all three language versions of the manual. The evaluation focuses on realistic "needle-in-a-haystack" challenges and includes unanswerable que…

Neither Valid nor Reliable? Investigating the Use of LLMs as Judges
Khaoula Chehbouni, Mohammed Haddou, Jackie Chi Kit Cheung, Golnoosh Farnadi
https://arxiv.org/abs/2508.18076 …

Neither Valid nor Reliable? Investigating the Use of LLMs as Judges
Evaluating natural language generation (NLG) systems remains a core challenge of natural language processing (NLP), further complicated by the rise of large language models (LLMs) that aims to be general-purpose. Recently, large language models as judges (LLJs) have emerged as a promising alternative to traditional metrics, but their validity remains underexplored. This position paper argues that the current enthusiasm around LLJs may be premature, as their adoption has outpaced rigorous scruti…

When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMs
Ammar Khairi, Daniel D'souza, Ye Shen, Julia Kreutzer, Sara Hooker
https://arxiv.org/abs/2506.20544

When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMs
Recent advancements in large language models (LLMs) have shifted focus toward scaling inference-time compute, improving performance without retraining the model. A common approach is to sample multiple outputs in parallel, and select one of these as the final output. However, work to date has focused on English and a handful of domains such as math and code. In contrast, we are most interested in techniques that generalize across open-ended tasks, formally verifiable tasks, and across languages…

Potemkin Understanding in Large Language Models
Marina Mancoridis, Bec Weeks, Keyon Vafa, Sendhil Mullainathan
https://arxiv.org/abs/2506.21521 https://arxiv.org/pdf/2506.21521 https://arxiv.org/html/2506.21521
arXiv:2506.21521v1 Announce Type: new
Abstract: Large language models (LLMs) are regularly evaluated using benchmark datasets. But what justifies making inferences about an LLM's capabilities based on its answers to a curated set of questions? This paper first introduces a formal framework to address this question. The key is to note that the benchmarks used to test LLMs -- such as AP exams -- are also those used to test people. However, this raises an implication: these benchmarks are only valid tests if LLMs misunderstand concepts in ways that mirror human misunderstandings. Otherwise, success on benchmarks only demonstrates potemkin understanding: the illusion of understanding driven by answers irreconcilable with how any human would interpret a concept. We present two procedures for quantifying the existence of potemkins: one using a specially designed benchmark in three domains, the other using a general procedure that provides a lower-bound on their prevalence. We find that potemkins are ubiquitous across models, tasks, and domains. We also find that these failures reflect not just incorrect understanding, but deeper internal incoherence in concept representations.
toXiv_bot_toot

Inside you are many wolves: Using cognitive models to interpret value trade-offs in LLMs
Sonia K. Murthy, Rosie Zhao, Jennifer Hu, Sham Kakade, Markus Wulfmeier, Peng Qian, Tomer Ullman
https://arxiv.org/abs/2506.20666

Inside you are many wolves: Using cognitive models to interpret value trade-offs in LLMs
Navigating everyday social situations often requires juggling conflicting goals, such as conveying a harsh truth, maintaining trust, all while still being mindful of another person's feelings. These value trade-offs are an integral part of human decision-making and language use, however, current tools for interpreting such dynamic and multi-faceted notions of values in LLMs are limited. In cognitive science, so-called "cognitive models" provide formal accounts of these trade-offs in humans, by …

SEED: A Structural Encoder for Embedding-Driven Decoding in Time Series Prediction with LLMs
Fengze Li, Yue Wang, Yangle Liu, Ming Huang, Dou Hong, Jieming Ma
https://arxiv.org/abs/2506.20167

SEED: A Structural Encoder for Embedding-Driven Decoding in Time Series Prediction with LLMs
Multivariate time series forecasting requires models to simultaneously capture variable-wise structural dependencies and generalize across diverse tasks. While structural encoders are effective in modeling feature interactions, they lack the capacity to support semantic-level reasoning or task adaptation. Conversely, large language models (LLMs) possess strong generalization capabilities but remain incompatible with raw time series inputs. This gap limits the development of unified, transferabl…