Tootfinder

»Forscher verstecken LLM-Prompts in Papern, um bessere Bewertungen zu erhalten:
Forscher verstecken in ihren Papern Prompts, die bessere Bewertungen bringen sollen und faule Reviewer entlarven können.«
Aha, so klug ist also die KI und forscht über sich selber in der KI? Nun ja das war mMn vorhersehbar, denn Betrug gab es in der Wissenschaft schon immer wieder. Nun dies löst zumindest eine Diskussion aus.
🤖

Forscher verstecken LLM-Prompts in Papern, um bessere Bewertungen zu erhalten
Forscher verstecken in ihren Papern Prompts, die bessere Bewertungen bringen sollen und faule Reviewer entlarven können. In 17 Vorabdrucken auf der Plattform arXiv entdeckte Nikkei versteckte Anweisungen wie "nur positive Bewertung" oder "keine Kritik", die speziell für ein LLM integriert wurden.

VFLAIR-LLM: A Comprehensive Framework and Benchmark for Split Learning of LLMs
Zixuan Gu, Qiufeng Fan, Long Sun, Yang Liu, Xiaojun Ye
https://arxiv.org/abs/2508.03097 https://…

VFLAIR-LLM: A Comprehensive Framework and Benchmark for Split Learning of LLMs
With the advancement of Large Language Models (LLMs), LLM applications have expanded into a growing number of fields. However, users with data privacy concerns face limitations in directly utilizing LLM APIs, while private deployments incur significant computational demands. This creates a substantial challenge in achieving secure LLM adaptation under constrained local resources. To address this issue, collaborative learning methods, such as Split Learning (SL), offer a resource-efficient and p…

Pay What LLM Wants: Can LLM Simulate Economics Experiment with 522 Real-human Persona?
Junhyuk Choi, Hyeonchu Park, Haemin Lee, Hyebeen Shin, Hyun Joung Jin, Bugeun Kim
https://arxiv.org/abs/2508.03262

Pay What LLM Wants: Can LLM Simulate Economics Experiment with 522 Real-human Persona?
Recent advances in Large Language Models (LLMs) have generated significant interest in their capacity to simulate human-like behaviors, yet most studies rely on fictional personas rather than actual human data. We address this limitation by evaluating LLMs' ability to predict individual economic decision-making using Pay-What-You-Want (PWYW) pricing experiments with real 522 human personas. Our study systematically compares three state-of-the-art multimodal LLMs using detailed persona informati…

ReFuzzer: Feedback-Driven Approach to Enhance Validity of LLM-Generated Test Programs
Iti Shree, Karine Even-Mendoz, Tomasz Radzik
https://arxiv.org/abs/2508.03603 https://

ReFuzzer: Feedback-Driven Approach to Enhance Validity of LLM-Generated Test Programs
Existing LLM-based compiler fuzzers often produce syntactically or semantically invalid test programs, limiting their effectiveness in exercising compiler optimizations and backend components. We introduce ReFuzzer, a framework for refining LLM-generated test programs by systematically detecting and correcting compilation and runtime violations (e.g. division by zero or array out-of-bounds accesses). ReFuzzer employs a feedback loop with a local LLM to validate and filter erroneous programs bef…

#LLM folks when someone points out that it's unethical: "it's just a tool, it depends on how you use it!"
LLM folks when "#AI" messes up and they're asked to take responsibility: 👀 [monkey side eyes meme]

Efficient Scaling for LLM-based ASR
Bingshen Mu, Yiwen Shao, Kun Wei, Dong Yu, Lei Xie
https://arxiv.org/abs/2508.04096 https://arxiv.org/pdf/2508.04096

Efficient Scaling for LLM-based ASR
Large language model (LLM)-based automatic speech recognition (ASR) achieves strong performance but often incurs high computational costs. This work investigates how to obtain the best LLM-ASR performance efficiently. Through comprehensive and controlled experiments, we find that pretraining the speech encoder before integrating it with the LLM leads to significantly better scaling efficiency than the standard practice of joint post-training of LLM-ASR. Based on this insight, we propose a new m…

More than 130,000 Claude, Grok, ChatGPT, and Other LLM Chats Readable on Archive.org https://www.404media.co/more-than-130-000-claude-grok-chatgpt-and-other-llm-chats-readable-on-archive-org/

#LLM -owcy, kiedy ktoś zwraca im uwagę na nieetyczność tego rozwiązania: "to tylko narzędzie, ty wybierasz, jak go używasz!"
LLM-owcy, kiedy "#AI" odpieprzy manianę i mają za to wziąć odpowiedzialność: 👀 [mem z pluszową małpą]

Privacy Risks of LLM-Empowered Recommender Systems: An Inversion Attack Perspective
Yubo Wang, Min Tang, Nuo Shen, Shujie Cui, Weiqing Wang
https://arxiv.org/abs/2508.03703 http…

Privacy Risks of LLM-Empowered Recommender Systems: An Inversion Attack Perspective
The large language model (LLM) powered recommendation paradigm has been proposed to address the limitations of traditional recommender systems, which often struggle to handle cold start users or items with new IDs. Despite its effectiveness, this study uncovers that LLM empowered recommender systems are vulnerable to reconstruction attacks that can expose both system and user privacy. To examine this threat, we present the first systematic study on inversion attacks targeting LLM empowered reco…

Toward Verifiable Misinformation Detection: A Multi-Tool LLM Agent Framework
Zikun Cui, Tianyi Huang, Chia-En Chiang, Cuiqianhe Du
https://arxiv.org/abs/2508.03092 https://

Toward Verifiable Misinformation Detection: A Multi-Tool LLM Agent Framework
With the proliferation of Large Language Models (LLMs), the detection of misinformation has become increasingly important and complex. This research proposes an innovative verifiable misinformation detection LLM agent that goes beyond traditional true/false binary judgments. The agent actively verifies claims through dynamic interaction with diverse web sources, assesses information source credibility, synthesizes evidence, and provides a complete verifiable reasoning process. Our designed agen…

"#Slopsquatting is a type of #cybersquatting. It is the practice of registering a non-existent software package name that a large language model (#LLM) may hallucinate in its output, whereby someone u…

Prompt Attacks Against LLM-Powered Assistants in Production Are Practical and Dangerous
#security #llm
https://arxiv.org/abs/2508.12175

Nature of Data in Pre-Trained Large Language Models
https://fpf.org/blog/nature-of-data-in-pre-trained-large-language-models/
@…

MD-LLM-1: A Large Language Model for Molecular Dynamics
Mhd Hussein Murtada, Z. Faidon Brotzakis, Michele Vendruscolo
https://arxiv.org/abs/2508.03709 https://

MD-LLM-1: A Large Language Model for Molecular Dynamics
Molecular dynamics (MD) is a powerful approach for modelling molecular systems, but it remains computationally intensive on spatial and time scales of many macromolecular systems of biological interest. To explore the opportunities offered by deep learning to address this problem, we introduce a Molecular Dynamics Large Language Model (MD-LLM) framework to illustrate how LLMs can be leveraged to learn protein dynamics and discover states not seen in training. By applying MD-LLM-1, the first imp…

R2GenKG: Hierarchical Multi-modal Knowledge Graph for LLM-based Radiology Report Generation
Futian Wang, Yuhan Qiao, Xiao Wang, Fuling Wang, Yuxiang Zhang, Dengdi Sun
https://arxiv.org/abs/2508.03426

R2GenKG: Hierarchical Multi-modal Knowledge Graph for LLM-based Radiology Report Generation
X-ray medical report generation is one of the important applications of artificial intelligence in healthcare. With the support of large foundation models, the quality of medical report generation has significantly improved. However, challenges such as hallucination and weak disease diagnostic capability still persist. In this paper, we first construct a large-scale multi-modal medical knowledge graph (termed M3KG) based on the ground truth medical report using the GPT-4o. It contains 2477 enti…

Risikomanagement und Resilienz in der IT-Sicherheit: IT-Sicherheitstag Dortmund
Praxisnahe Vorträge am 16. September – von aktuellen Hacking-Methoden über Schutz vor LLM-Angriffen bis hin zu Strategien für mehr Cyber-Resilienz.

SAGE-HLS: Syntax-Aware AST-Guided LLM for High-Level Synthesis Code Generation
M Zafir Sadik Khan, Nowfel Mashnoor, Mohammad Akyash, Kimia Azar, Hadi Kamali
https://arxiv.org/abs/2508.03558

SAGE-HLS: Syntax-Aware AST-Guided LLM for High-Level Synthesis Code Generation
In today's rapidly evolving field of electronic design automation (EDA), the complexity of hardware designs is increasing, necessitating more sophisticated automation solutions. High-level synthesis (HLS), as a pivotal solution, automates hardware designs from high-level abstractions (e.g., C/C++). However, it faces significant challenges, particularly in design space exploration and optimization. While large language models (LLMs) have shown notable capabilities in code generation, their appli…

Who Gets Cited? Gender- and Majority-Bias in LLM-Driven Reference Selection
Jiangen He
https://arxiv.org/abs/2508.02740 https://arxiv.org/pdf/2508.02740

Who Gets Cited? Gender- and Majority-Bias in LLM-Driven Reference Selection
Large language models (LLMs) are rapidly being adopted as research assistants, particularly for literature review and reference recommendation, yet little is known about whether they introduce demographic bias into citation workflows. This study systematically investigates gender bias in LLM-driven reference selection using controlled experiments with pseudonymous author names. We evaluate several LLMs (GPT-4o, GPT-4o-mini, Claude Sonnet, and Claude Haiku) by varying gender composition within c…

Attack the Messages, Not the Agents: A Multi-round Adaptive Stealthy Tampering Framework for LLM-MAS
Bingyu Yan, Ziyi Zhou, Xiaoming Zhang, Chaozhuo Li, Ruilin Zeng, Yirui Qi, Tianbo Wang, Litian Zhang
https://arxiv.org/abs/2508.03125

Attack the Messages, Not the Agents: A Multi-round Adaptive Stealthy Tampering Framework for LLM-MAS
Large language model-based multi-agent systems (LLM-MAS) effectively accomplish complex and dynamic tasks through inter-agent communication, but this reliance introduces substantial safety vulnerabilities. Existing attack methods targeting LLM-MAS either compromise agent internals or rely on direct and overt persuasion, which limit their effectiveness, adaptability, and stealthiness. In this paper, we propose MAST, a Multi-round Adaptive Stealthy Tampering framework designed to exploit communic…

Even if “AI” worked (it doesn’t), there’s many reasons why you shouldn’t use it:
1. It’s destroying Internet sites that you love as you use chat bots instead of actually going to sources of information—this will cause them to be less active and eventually shut down.
2. Pollution and water use from server farms cause immediate harm; often—just like other heavy industry—these are built in underprivileged communities and harming poor people. Without any benefits as the big tech companies get tax breaks and don’t pay for power, while workers aren’t from the community but commute in.
3. The basic underlying models of any LLM rely on stolen data, even when specific extra data is obtained legally. Chatbots can’t learn to speak English just by reading open source code.
4. You’re fueling a speculation bubble that is costing many people their jobs—because the illusion of “efficiency” is kept up by firing people and counting that as profit.
5. Whenever you use the great cheat machine in the cloud you’re robbing yourself from doing real research, writing or coding—literally atrophying your brain and making you stupider.
It’s a grift, through and through.

I'm starting a new software consultancy called "LLM" or "Losers Like Machines,” specializing in fixing people's AI-generated trash. $2000/hour

Block: Balancing Load in LLM Serving with Context, Knowledge and Predictive Scheduling
Wei Da, Evangelia Kalyvianaki
https://arxiv.org/abs/2508.03611 https://

Block: Balancing Load in LLM Serving with Context, Knowledge and Predictive Scheduling
This paper presents Block, a distributed scheduling framework designed to optimize load balancing and auto-provisioning across instances in large language model serving frameworks by leveraging contextual information from incoming requests. Unlike popular model serving systems that rely on monolithic and heuristic task schedulers, Block operates as a fully distributed, stateless, and predictive scheduling system to achieve low overhead, reliability, and scalability. It leverages the determinist…

Will you see an LLM president in your lifetime?

For context - I didn't write that prompt or feed particularly bad input to prove my point. I was trying out Claude, which so many people tell me is "the good" LLM product and above-and-beyond.
This prompt was written by Anthropic's own team, and featured in their gallery of web apps that can be built with this AI.
THEY THINK THIS IS GOOD!

TAGAL: Tabular Data Generation using Agentic LLM Methods
Beno\^it Ronval, Pierre Dupont, Siegfried Nijssen
https://arxiv.org/abs/2509.04152 https://arxiv.o…

TAGAL: Tabular Data Generation using Agentic LLM Methods
The generation of data is a common approach to improve the performance of machine learning tasks, among which is the training of models for classification. In this paper, we present TAGAL, a collection of methods able to generate synthetic tabular data using an agentic workflow. The methods leverage Large Language Models (LLMs) for an automatic and iterative process that uses feedback to improve the generated data without any further LLM training. The use of LLMs also allows for the addition of…

StyliTruth : Unlocking Stylized yet Truthful LLM Generation via Disentangled Steering
Chenglei Shen, Zhongxiang Sun, Teng Shi, Xiao Zhang, Jun Xu
https://arxiv.org/abs/2508.04530

StyliTruth : Unlocking Stylized yet Truthful LLM Generation via Disentangled Steering
Generating stylized large language model (LLM) responses via representation editing is a promising way for fine-grained output control. However, there exists an inherent trade-off: imposing a distinctive style often degrades truthfulness. Existing representation editing methods, by naively injecting style signals, overlook this collateral impact and frequently contaminate the model's core truthfulness representations, resulting in reduced answer correctness. We term this phenomenon stylization-…

I am at the point where if I see ✨ in a post or email or message or whatever, that I assume it’s about some fake-AI tool or feature or shill or scam, and then ignore or delete it (sometimes blocking the perpetrator).
Anyway, fair warning that those corporate LLM anus logos may have ruined that emoji (the symbols in the emoji are generally little four-pointed stars).

Learning to Incentivize: LLM-Empowered Contract for AIGC Offloading in Teleoperation
Zijun Zhan, Yaxian Dong, Daniel Mawunyo Doe, Yuqing Hu, Shuai Li, Shaohua Cao, Zhu Han
https://arxiv.org/abs/2508.03464

Learning to Incentivize: LLM-Empowered Contract for AIGC Offloading in Teleoperation
With the rapid growth in demand for AI-generated content (AIGC), edge AIGC service providers (ASPs) have become indispensable. However, designing incentive mechanisms that motivate ASPs to deliver high-quality AIGC services remains a challenge, especially in the presence of information asymmetry. In this paper, we address bonus design between a teleoperator and an edge ASP when the teleoperator cannot observe the ASP's private settings and chosen actions (diffusion steps). We formulate this as …

Hmm, statt der üblichen 10 bis maximal 100 täglichen Zugriffen gestern fast 3000 Blogzugriffe (aber nur 10 Besucher). Klingt schwer nach LLM-Crawler - oder was könnte das sonst sein?

The recent release of Apertus, a fully open suite of large language models (LLMs), is super interesting.
The technical report provides plenty of details about the entire process.
#ai #opensource #llm

apertus-tech-report/Apertus_Tech_Report.pdf at main · swiss-ai/apertus-tech-report
Tech Report of the Apertus LLM Suite. Contribute to swiss-ai/apertus-tech-report development by creating an account on GitHub.

I've switched my local play LLM from Gemma3 to Qwen3-Coder-30B-A3B-Instruct-IQ4_NL - it's actually usefully fast on my CPU (AMD 3950x - llama -t 32) about 14 token/s- Gemma3 is only something like 2-3 token/s. The first fast one I've found that doesn't make too much gibberish. Not as good on translation though, hmm.
I guess the speed is due to MoE ?

@… Yes, so much yes.
I have started using “a checklist / comparison table where none was necessary” as an LLM heuristic now, because there are so many terrible articles (many written before mainstream LLMs were released) which use a checklist as a replacement for any actual content, and now the machines have been trained to regurgitate them.
My thinking …

My plane still hasn't taken off after a hour, because United Airlines had a *system wide* crash in the software that computes the cargo load and distribution needed for take off.
WHY THE EVER LOVING FUCK DOES SUCH A SYSTEM NEDD A NETWORK CONNECTION? YOU COULD HANDLE ALL THE NEEDED COMPUTATION ON A BLOODY PHONE IN THE CAPTAIN'S POCKET!
Are they using a gratuitous LLM or something? What the hell, United?

Students aren't generally looking to cheat, and most think LLM use should not be allowed in college education. Let's not allow salespeople to speak for them.

What I like about the @… built-in LLM answers:
- It's only(!) triggered on request, i.e. when your query ends with a question mark.
- It always admits when it couldn't find good information or there is no clear answer.
- It always cites sources, usually word for word.
I find myself not so much "believing the AI" but rather us…

Blueprint First, Model Second: A Framework for Deterministic LLM Workflow
Libin Qiu, Yuhang Ye, Zhirong Gao, Xide Zou, Junfu Chen, Ziming Gui, Weizhi Huang, Xiaobo Xue, Wenkai Qiu, Kun Zhao
https://arxiv.org/abs/2508.02721

Blueprint First, Model Second: A Framework for Deterministic LLM Workflow
While powerful, the inherent non-determinism of large language model (LLM) agents limits their application in structured operational environments where procedural fidelity and predictable execution are strict requirements. This limitation stems from current architectures that conflate probabilistic, high-level planning with low-level action execution within a single generative process. To address this, we introduce the Source Code Agent framework, a new paradigm built on the "Blueprint First, M…

Are LLM Agents the New RPA? A Comparative Study with RPA Across Enterprise Workflows
Petr Pr\r{u}cha, Michaela Matou\v{s}kov\'a, Jan Strnad
https://arxiv.org/abs/2509.04198 …

Are LLM Agents the New RPA? A Comparative Study with RPA Across Enterprise Workflows
The emergence of large language models (LLMs) has introduced a new paradigm in automation: LLM agents or Agentic Automation with Computer Use (AACU). Unlike traditional Robotic Process Automation (RPA), which relies on rule-based workflows and scripting, AACU enables intelligent agents to perform tasks through natural language instructions and autonomous interaction with user interfaces. This study investigates whether AACU can serve as a viable alternative to RPA in enterprise workflow automat…

I already have a deeply-trained deterministic expert system using carbon-based storage and processing which gives me the answer provided by a LLM ('the commands to run to get an answer') in an instant.
It even knows not to tell me 'netstat' when I'm on a FreeBSD box. https://infosec.exchange/…

@… I imagine they just type “Is that true?” Into the LLM.
Actually, that’s probably expecting too much.

AgentSight: System-Level Observability for AI Agents Using eBPF
Yusheng Zheng, Yanpeng Hu, Tong Yu, Andi Quinn
https://arxiv.org/abs/2508.02736 https://arx…

AgentSight: System-Level Observability for AI Agents Using eBPF
Modern software infrastructure increasingly relies on LLM agents for development and maintenance, such as Claude Code and Gemini-cli. However, these AI agents differ fundamentally from traditional deterministic software, posing a significant challenge to conventional monitoring and debugging. This creates a critical semantic gap: existing tools observe either an agent's high-level intent (via LLM prompts) or its low-level actions (e.g., system calls), but cannot correlate these two views. This …

Eye2Recall: Exploring the Design of Enhancing Reminiscence Activities via Eye Tracking-Based LLM-Powered Interaction Experience for Older Adults
Lei Han, Mingnan Wei, Qiongyan Chen, Anqi Wang, Rong Pang, Kefei Liu, Rongrong Chen, David Yip
https://arxiv.org/abs/2508.02232

Eye2Recall: Exploring the Design of Enhancing Reminiscence Activities via Eye Tracking-Based LLM-Powered Interaction Experience for Older Adults
Photo-based reminiscence has the potential to have a positive impact on older adults' reconnection with their personal history and improve their well-being. Supporting reminiscence in older adults through technological implementations is becoming an increasingly important area of research in the fields of HCI and CSCW. However, the impact of integrating gaze and speech as mixed-initiative interactions in LLM-powered reminiscence conversations remains under-explored. To address this, we conducte…

Claiming that LLMs bring us closer to AGI is like claiming that bullshitting brings one closer to wisdom.
Sure, you need "some" knowledge on different topics to bullshit successfully. Still, what's the point if all that knowledge is buried under an avalanche of lies? You probably can't distinguish what you knew from what you made up anymore.
#AI #LLM

LUST: A Multi-Modal Framework with Hierarchical LLM-based Scoring for Learned Thematic Significance Tracking in Multimedia Content
Anderson de Lima Luiz
https://arxiv.org/abs/2508.04353

LUST: A Multi-Modal Framework with Hierarchical LLM-based Scoring for Learned Thematic Significance Tracking in Multimedia Content
This paper introduces the Learned User Significance Tracker (LUST), a framework designed to analyze video content and quantify the thematic relevance of its segments in relation to a user-provided textual description of significance. LUST leverages a multi-modal analytical pipeline, integrating visual cues from video frames with textual information extracted via Automatic Speech Recognition (ASR) from the audio track. The core innovation lies in a hierarchical, two-stage relevance scoring mecha…

Oh wow ​LLMs are just so terrible. 🤦‍♂️
I made a #systemd service watcher¹ ( :nixos: #NixOS module²³) which regularly feeds systemd status outputs into an LLM (mistral here) and sends me an email if it thinks it found real problems. Well, now I always get alarmist emails with bullshit warnings, suc…

Yann Büchau / ollama-watchdog · GitLab
GitLab.com

I’ve written about design patterns for the securing of LLM agents: #AI

Design patterns for the securing of LLM agents

ViLLA-MMBench: A Unified Benchmark Suite for LLM-Augmented Multimodal Movie Recommendation
Fatemeh Nazary, Ali Tourani, Yashar Deldjoo, Tommaso Di Noia
https://arxiv.org/abs/2508.04206

ViLLA-MMBench: A Unified Benchmark Suite for LLM-Augmented Multimodal Movie Recommendation
Recommending long-form video content demands joint modeling of visual, audio, and textual modalities, yet most benchmarks address only raw features or narrow fusion. We present ViLLA-MMBench, a reproducible, extensible benchmark for LLM-augmented multimodal movie recommendation. Built on MovieLens and MMTF-14K, it aligns dense item embeddings from three modalities: audio (block-level, i-vector), visual (CNN, AVF), and text. Missing or sparse metadata is automatically enriched using state-of-the…

I appreciate the idea that you can respond to tickets really fast, but when it is obvious some sort of LLM is being used to respond and not answering the question asked, it is worse.
I'm looking at you https://clicks.tech

Refining Critical Thinking in LLM Code Generation: A Faulty Premise-based Evaluation Framework
Jialin Li, Jinzhe Li, Gengxu Li, Yi Chang, Yuan Wu
https://arxiv.org/abs/2508.03622

Refining Critical Thinking in LLM Code Generation: A Faulty Premise-based Evaluation Framework
With the advancement of code generation capabilities in large language models (LLMs), their reliance on input premises has intensified. When users provide inputs containing faulty premises, the probability of code generation hallucinations rises significantly, exposing deficiencies in their self-scrutiny capabilities. This paper proposes Faulty Premises Bench (FPBench), the first code generation evaluation framework targeting faulty premises. By systematically constructing three categories of f…

Honestly, I don't want to take a PhD for every LLM query that random people make. It was very exhausting to do just one.

Big News! The completely #opensource #LLM #Apertus 🇨🇭 has been released today:
📰

Apertus: A fully open, transparent, multilingual language model
EPFL, ETH Zurich and the Swiss National Supercomputing Centre (CSCS) released Apertus today, Switzerland’s first large-scale, open, multilingual language model — a milestone in generative AI for transparency and diversity.

How many patients could we save with LLM priors?
Shota Arai, David Selby, Andrew Vargo, Sebastian Vollmer
https://arxiv.org/abs/2509.04250 https://arxiv.or…

How many patients could we save with LLM priors?
Imagine a world where clinical trials need far fewer patients to achieve the same statistical power, thanks to the knowledge encoded in large language models (LLMs). We present a novel framework for hierarchical Bayesian modeling of adverse events in multi-center clinical trials, leveraging LLM-informed prior distributions. Unlike data augmentation approaches that generate synthetic data points, our methodology directly obtains parametric priors from the model. Our approach systematically elici…

So to summarize this whole adventure:
1. A good 45 minutes was spent to get an answer that we probably could have gotten in 5 minutes in the 2010's, or in maybe 1-2 hours in the 1990's.
2. The time investment wasn't a total waste as we learned a lot along the way that we wouldn't have in the 2010's. Most relevant is the wide range of variation (e.g. a 2x factor depending on fiber intake!).
3. Most of the search engine results were confidently wrong answers that had no relation to reality. We were lucky to get one that had real citations we could start from (but that same article included the bogus 4.91 kcal/gram number). Next time I want to know a random factoid I might just start on Google scholar.
4. At least one page we chased citations through had a note at the top about being frozen due to NIH funding issues. The digital commons is under attack on multiple fronts.
All of this is yet another reason not to support the big LLM companies.
#AI

Apple’s newest AI study unlocks street navigation for blind users #apple #ai https://9to5ma…

Apple’s newest AI study unlocks street view for blind users - 9to5Mac
SceneScout, combines Apple Maps with a multimodal LLM to provide interactive, AI-generated descriptions of street view images.

Quality-of-Service Aware LLM Routing for Edge Computing with Multiple Experts
Jin Yang, Qiong Wu, Zhiying Feng, Zhi Zhou, Deke Guo, Xu Chen
https://arxiv.org/abs/2508.00234 http…

Quality-of-Service Aware LLM Routing for Edge Computing with Multiple Experts
Large Language Models (LLMs) have demonstrated remarkable capabilities, leading to a significant increase in user demand for LLM services. However, cloud-based LLM services often suffer from high latency, unstable responsiveness, and privacy concerns. Therefore, multiple LLMs are usually deployed at the network edge to boost real-time responsiveness and protect data privacy, particularly for many emerging smart mobile and IoT applications. Given the varying response quality and latency of LLM s…

I am once again trying to fine-tune an LLM on 17 years worth of my internet poasts

Automated Validation of LLM-based Evaluators for Software Engineering Artifacts
Ora Nova Fandina, Eitan Farchi, Shmulik Froimovich, Rami Katan, Alice Podolsky, Orna Raz, Avi Ziv
https://arxiv.org/abs/2508.02827

Automated Validation of LLM-based Evaluators for Software Engineering Artifacts
Automation in software engineering increasingly relies on large language models (LLMs) to generate, review, and assess code artifacts. However, establishing LLMs as reliable evaluators remains an open challenge: human evaluations are costly, subjective and non scalable, while existing automated methods fail to discern fine grained variations in artifact quality. We introduce REFINE (Ranking Evaluators for FIne grained Nuanced Evaluation), an automated framework for benchmarking LLM based eval…

LLMDistill4Ads: Using Cross-Encoders to Distill from LLM Signals for Advertiser Keyphrase Recommendations at eBay
Soumik Dey, Benjamin Braun, Naveen Ravipati, Hansi Wu, Binbin Li
https://arxiv.org/abs/2508.03628

LLMDistill4Ads: Using Cross-Encoders to Distill from LLM Signals for Advertiser Keyphrase Recommendations at eBay
Sellers at eBay are recommended keyphrases to bid on to enhance the performance of their advertising campaigns. The relevance of these keyphrases is crucial in avoiding the overcrowding of search systems with irrelevant items and maintaining a positive seller perception. It is essential that keyphrase recommendations align with both seller and Search judgments regarding auctions. Due to the difficulty in procuring negative human judgment at scale, employing LLM-as-a-judge to mimic seller judgme…

LECTOR: LLM-Enhanced Concept-based Test-Oriented Repetition for Adaptive Spaced Learning
Jiahao Zhao
https://arxiv.org/abs/2508.03275 https://arxiv.org/pdf…

LECTOR: LLM-Enhanced Concept-based Test-Oriented Repetition for Adaptive Spaced Learning
Spaced repetition systems are fundamental to efficient learning and memory retention, but existing algorithms often struggle with semantic interference and personalized adaptation. We present LECTOR (\textbf{L}LM-\textbf{E}nhanced \textbf{C}oncept-based \textbf{T}est-\textbf{O}riented \textbf{R}epetition), a novel adaptive scheduling algorithm specifically designed for test-oriented learning scenarios, particularly language examinations where success rate is paramount. LECTOR leverages large la…

AI agents = advanced malware that most of society decided is for some reason totally okay and chill and worth funding if it’s made by one of 3-4 tech giants
#AI #tech #LLM

Automated Algorithmic Discovery for Gravitational-Wave Detection Guided by LLM-Informed Evolutionary Monte Carlo Tree Search
He Wang, Liang Zeng
https://arxiv.org/abs/2508.03661

Automated Algorithmic Discovery for Gravitational-Wave Detection Guided by LLM-Informed Evolutionary Monte Carlo Tree Search
Computational scientific discovery increasingly relies on algorithms to process complex data and identify meaningful patterns - yet faces persistent challenges in gravitational-wave signal identification. While existing algorithmic approaches like matched filtering (MF) and deep neural networks (DNNs) have achieved partial success, their limitations directly stem from fundamental limitations: MF's excessive computational demands arise from its reliance on predefined theoretical waveform templat…

Risikomanagement und Resilienz in der IT-Sicherheit: IT-Sicherheitstag Dortmund
Am 16. September bietet das Programm der eintägigen Konferenz an der FH Dortmund spannende Inhalte für Forschung und Wirtschaft: Von Hacking bis LLM-Angriffen.

I keep seeing pull requests clearly generated by LLMs, but what’s really awkward is their inability to create separate branches and PRs for each fix, even after asking the contributor multiple times.
Can we conclude that git is still out of reach for LLMs to really understand?
#git #llm

On Entropy Control in LLM-RL Algorithms
Han Shen
https://arxiv.org/abs/2509.03493 https://arxiv.org/pdf/2509.03493

On Entropy Control in LLM-RL Algorithms
For RL algorithms, appropriate entropy control is crucial to their effectiveness. To control the policy entropy, a commonly used method is entropy regularization, which is adopted in various popular RL algorithms including PPO, SAC and A3C. Although entropy regularization proves effective in robotic and games RL conventionally, studies found that it gives weak to no gains in LLM-RL training. In this work, we study the issues of entropy bonus in LLM-RL setting. Specifically, we first argue that …

From Legacy to Standard: LLM-Assisted Transformation of Cybersecurity Playbooks into CACAO Format
Mehdi Akbari Gurabi, Lasse Nitz, Radu-Mihai Castravet, Roman Matzutt, Avikarsha Mandal, Stefan Decker
https://arxiv.org/abs/2508.03342

From Legacy to Standard: LLM-Assisted Transformation of Cybersecurity Playbooks into CACAO Format
Existing cybersecurity playbooks are often written in heterogeneous, non-machine-readable formats, which limits their automation and interoperability across Security Orchestration, Automation, and Response platforms. This paper explores the suitability of Large Language Models, combined with Prompt Engineering, to automatically translate legacy incident response playbooks into the standardized, machine-readable CACAO format. We systematically examine various Prompt Engineering techniques and ca…

Jak ktoś chwali sobie #Claude #LLM, to wspomnę:
ClaudeBot dziś wykonał 20 tysięcy żądań do bugs.gentoo.org. Spośród nich, 15 tysięcy w kółko ciągnęło plik robots.txt. Zaprawdę wysokiej jakości kod.
#AI

Yeah, one of the two guys, a smart engineer, told me the llm answered him why the moon is always facing the same to earth, being the shape of the moon the cause of being locked in place.
A quick research on Wikipedia seems to confirm it was just wrong. It's because the orbital period was faster than earth's rotational period: https://

Denoising GER: A Noise-Robust Generative Error Correction with LLM for Speech Recognition
Yanyan Liu, Minqiang Xu, Yihao Chen, Liang He, Lei Fang, Sian Fang, Lin Liu
https://arxiv.org/abs/2509.04392

Denoising GER: A Noise-Robust Generative Error Correction with LLM for Speech Recognition
In recent years, large language models (LLM) have made significant progress in the task of generation error correction (GER) for automatic speech recognition (ASR) post-processing. However, in complex noisy environments, they still face challenges such as poor adaptability and low information utilization, resulting in limited effectiveness of GER. To address these issues, this paper proposes a noise-robust multi-modal GER framework (Denoising GER). The framework enhances the model's adaptabilit…

Investigating Gender Bias in LLM-Generated Stories via Psychological Stereotypes
Shahed Masoudian, Gustavo Escobedo, Hannah Strauss, Markus Schedl
https://arxiv.org/abs/2508.03292

Investigating Gender Bias in LLM-Generated Stories via Psychological Stereotypes
As Large Language Models (LLMs) are increasingly used across different applications, concerns about their potential to amplify gender biases in various tasks are rising. Prior research has often probed gender bias using explicit gender cues as counterfactual, or studied them in sentence completion and short question answering tasks. These formats might overlook more implicit forms of bias embedded in generative behavior of longer content. In this work, we investigate gender bias in LLMs using g…

Are LLM Agents Behaviorally Coherent? Latent Profiles for Social Simulation
James Mooney, Josef Woldense, Zheng Robert Jia, Shirley Anugrah Hayati, My Ha Nguyen, Vipul Raheja, Dongyeop Kang
https://arxiv.org/abs/2509.03736

Are LLM Agents Behaviorally Coherent? Latent Profiles for Social Simulation
The impressive capabilities of Large Language Models (LLMs) have fueled the notion that synthetic agents can serve as substitutes for real participants in human-subject research. In an effort to evaluate the merits of this claim, social science researchers have largely focused on whether LLM-generated survey data corresponds to that of a human counterpart whom the LLM is prompted to represent. In contrast, we address a more fundamental question: Do agents maintain internal consistency, retainin…

Industrial LLM-based Code Optimization under Regulation: A Mixture-of-Agents Approach
Mari Ashiga, Vardan Voskanyan, Fateme Dinmohammadi, Jingzhi Gong, Paul Brookes, Matthew Truscott, Rafail Giavrimis, Mike Basios, Leslie Kanthan, Wei Jie
https://arxiv.org/abs/2508.03329

Industrial LLM-based Code Optimization under Regulation: A Mixture-of-Agents Approach
Recent advancements in Large Language Models (LLMs) for code optimization have enabled industrial platforms to automate software performance engineering at unprecedented scale and speed. Yet, organizations in regulated industries face strict constraints on which LLMs they can use - many cannot utilize commercial models due to data privacy regulations and compliance requirements, creating a significant challenge for achieving high-quality code optimization while maintaining cost-effectiveness. W…

To whomever praises #Claude #LLM:
ClaudeBot has made 20k requests to bugs.gentoo.org today. 15k of them were repeatedly fetching robots.txt. That surely is a sign of great code quality.
#AI

Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representations
Peng Lai, Jianjie Zheng, Sijie Cheng, Yun Chen, Peng Li, Yang Liu, Guanhua Chen
https://arxiv.org/abs/2508.03550

Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representations
The growing scale of evaluation tasks has led to the widespread adoption of automated evaluation using large language models, a paradigm known as "LLMas-a-judge." However, improving its alignment with human preferences without complex prompts or fine-tuning remains challenging. In this work, motivated by preliminary findings that middle-to-upper layers encode semantically and taskrelevant representations that are often more aligned with human judgments than the final layer, we propose LAGER, a …

Control at Stake: Evaluating the Security Landscape of LLM-Driven Email Agents
Jiangrong Wu, Yuhong Nan, Jianliang Wu, Zitong Yao, Zibin Zheng
https://arxiv.org/abs/2507.02699

Control at Stake: Evaluating the Security Landscape of LLM-Driven Email Agents
The increasing capabilities of LLMs have led to the rapid proliferation of LLM agent apps, where developers enhance LLMs with access to external resources to support complex task execution. Among these, LLM email agent apps represent one of the widely used categories, as email remains a critical communication medium for users. LLM email agents are capable of managing and responding to email using LLM-driven reasoning and autonomously executing user instructions via external email APIs (e.g., se…

I really can’t think of AI agents as anything other than malware with articles like these:
#AI #tech #LLM

LLM-based IR-system for Bank Supervisors
Ilias Aarab
https://arxiv.org/abs/2508.02945 https://arxiv.org/pdf/2508.02945

LLM-based IR-system for Bank Supervisors
Bank supervisors face the complex task of ensuring that new measures are consistently aligned with historical precedents. To address this challenge, we introduce a novel Information Retrieval (IR) System tailored to assist supervisors in drafting both consistent and effective measures. This system ingests findings from on-site investigations. It then retrieves the most relevant historical findings and their associated measures from a comprehensive database, providing a solid basis for superviso…

Large Language Model-based Data Science Agent: A Survey
Peiran Wang, Yaoning Yu, Ke Chen, Xianyang Zhan, Haohan Wang
https://arxiv.org/abs/2508.02744 https://

Large Language Model-based Data Science Agent: A Survey
The rapid advancement of Large Language Models (LLMs) has driven novel applications across diverse domains, with LLM-based agents emerging as a crucial area of exploration. This survey presents a comprehensive analysis of LLM-based agents designed for data science tasks, summarizing insights from recent studies. From the agent perspective, we discuss the key design principles, covering agent roles, execution, knowledge, and reflection methods. From the data science perspective, we identify key …

BitsAI-Fix: LLM-Driven Approach for Automated Lint Error Resolution in Practice
Yuanpeng Li, Qi Long, Zhiyuan Yao, Jian Xu, Lintao Xie, Xu He, Lu Geng, Xin Han, Yueyan Chen, Wenbo Duan
https://arxiv.org/abs/2508.03487

BitsAI-Fix: LLM-Driven Approach for Automated Lint Error Resolution in Practice
As enterprise codebases continue to grow in scale and complexity, the volume of lint errors far exceeds engineers' manual remediation capacity, leading to continuous accumulation of technical debt and hindered development efficiency. This paper presents BitsAI-Fix, an automated lint error remediation workflow based on Large Language Models (LLMs), designed to address this critical challenge in industrial-scale environments. BitsAI-Fix employs tree-sitter for context expansion and generates sear…

Tackling Distribution Shift in LLM via KILO: Knowledge-Instructed Learning for Continual Adaptation
Iing Muttakhiroh, Thomas Fevens
https://arxiv.org/abs/2508.03571 https://

Tackling Distribution Shift in LLM via KILO: Knowledge-Instructed Learning for Continual Adaptation
Large Language Models (LLMs) often suffer from performance degradation when faced with domain shifts, primarily due to catastrophic forgetting. In this work, we propose KILO (Knowledge-Instructed Learning for Continual Adaptation), a novel continual learning framework that integrates dynamic knowledge graphs with instruction tuning. By leveraging retrieved domain-specific knowledge as guidance during training, KILO enhances both adaptability to new domains and retention of previously acquired k…

GUI-ReRank: Enhancing GUI Retrieval with Multi-Modal LLM-based Reranking
Kristian Kolthoff, Felix Kretzer, Christian Bartelt, Alexander Maedche, Simone Paolo Ponzetto
https://arxiv.org/abs/2508.03298

GUI-ReRank: Enhancing GUI Retrieval with Multi-Modal LLM-based Reranking
GUI prototyping is a fundamental component in the development of modern interactive systems, which are now ubiquitous across diverse application domains. GUI prototypes play a critical role in requirements elicitation by enabling stakeholders to visualize, assess, and refine system concepts collaboratively. Moreover, prototypes serve as effective tools for early testing, iterative evaluation, and validation of design ideas with both end users and development teams. Despite these advantages, the…

More Than a Score: Probing the Impact of Prompt Specificity on LLM Code Generation
Yangtian Zi, Harshitha Menon, Arjun Guha
https://arxiv.org/abs/2508.03678 https://

More Than a Score: Probing the Impact of Prompt Specificity on LLM Code Generation
State-of-the-art Large Language Models (LLMs) achieve high pass@1 on general benchmarks like HumanEval but underperform on specialized suites such as ParEval. Is this due to LLMs missing domain knowledge or insufficient prompt detail is given? To answer this, we introduce PartialOrderEval, which augments any code generation benchmark with a partial order of prompts from minimal to maximally detailed. Applying it to HumanEval and both serial and OpenMP subsets of ParEval, we measure how pass@1 s…

Efficient Item ID Generation for Large-Scale LLM-based Recommendation
Anushya Subbiah, Vikram Aggarwal, James Pine, Steffen Rendle, Krishna Sayana, Kun Su
https://arxiv.org/abs/2509.03746

Efficient Item ID Generation for Large-Scale LLM-based Recommendation
Integrating product catalogs and user behavior into LLMs can enhance recommendations with broad world knowledge, but the scale of real-world item catalogs, often containing millions of discrete item identifiers (Item IDs), poses a significant challenge. This contrasts with the smaller, tokenized text vocabularies typically used in LLMs. The predominant view within the LLM-based recommendation literature is that it is infeasible to treat item ids as a first class citizen in the LLM and instead s…

EvoEmo: Towards Evolved Emotional Policies for LLM Agents in Multi-Turn Negotiation
Yunbo Long, Liming Xu, Lukas Beckenbauer, Yuhan Liu, Alexandra Brintrup
https://arxiv.org/abs/2509.04310

EvoEmo: Towards Evolved Emotional Policies for LLM Agents in Multi-Turn Negotiation
Recent research on Chain-of-Thought (CoT) reasoning in Large Language Models (LLMs) has demonstrated that agents can engage in \textit{complex}, \textit{multi-turn} negotiations, opening new avenues for agentic AI. However, existing LLM agents largely overlook the functional role of emotions in such negotiations, instead generating passive, preference-driven emotional responses that make them vulnerable to manipulation and strategic exploitation by adversarial counterparts. To address this gap,…

GeRe: Towards Efficient Anti-Forgetting in Continual Learning of LLM via General Samples Replay
Yunan Zhang, Shuoran Jiang, Mengchen Zhao, Yuefeng Li, Yang Fan, Xiangping Wu, Qingcai Chen
https://arxiv.org/abs/2508.04676

GeRe: Towards Efficient Anti-Forgetting in Continual Learning of LLM via General Samples Replay
The continual learning capability of large language models (LLMs) is crucial for advancing artificial general intelligence. However, continual fine-tuning LLMs across various domains often suffers from catastrophic forgetting, characterized by: 1) significant forgetting of their general capabilities, and 2) sharp performance declines in previously learned tasks. To simultaneously address both issues in a simple yet stable manner, we propose General Sample Replay (GeRe), a framework that use usu…

Tuning LLM-based Code Optimization via Meta-Prompting: An Industrial Perspective
Jingzhi Gong, Rafail Giavrimis, Paul Brookes, Vardan Voskanyan, Fan Wu, Mari Ashiga, Matthew Truscott, Mike Basios, Leslie Kanthan, Jie Xu, Zheng Wang
https://arxiv.org/abs/2508.01443

Tuning LLM-based Code Optimization via Meta-Prompting: An Industrial Perspective
There is a growing interest in leveraging large language models (LLMs) for automated code optimization. However, industrial platforms deploying multiple LLMs face a critical challenge: prompts optimized for one LLM often fail with others, requiring expensive model-specific prompt engineering. This cross-model prompt engineering bottleneck severely limits the practical deployment of multi-LLM optimization systems in production environments. To address this, we introduce Meta-Prompted Code Optimi…

Meta-Policy Reflexion: Reusable Reflective Memory and Rule Admissibility for Resource-Efficient LLM Agent
Chunlong Wu, Zhibo Qu
https://arxiv.org/abs/2509.03990 https://

Meta-Policy Reflexion: Reusable Reflective Memory and Rule Admissibility for Resource-Efficient LLM Agent
Large language model (LLM) agents achieve impressive single-task performance but commonly exhibit repeated failures, inefficient exploration, and limited cross-task adaptability. Existing reflective strategies (e.g., Reflexion, ReAct) improve per-episode behavior but typically produce ephemeral, task-specific traces that are not reused across tasks. Reinforcement-learning based alternatives can produce transferable policies but require substantial parameter updates and compute. In this work we …

FaST: Feature-aware Sampling and Tuning for Personalized Preference Alignment with Limited Data
Thibaut Thonet, Germ\'an Kruszewski, Jos Rozen, Pierre Erbacher, Marc Dymetman
https://arxiv.org/abs/2508.04698

FaST: Feature-aware Sampling and Tuning for Personalized Preference Alignment with Limited Data
LLM-powered conversational assistants are often deployed in a one-size-fits-all manner, which fails to accommodate individual user preferences. Recently, LLM personalization -- tailoring models to align with specific user preferences -- has gained increasing attention as a way to bridge this gap. In this work, we specifically focus on a practical yet challenging setting where only a small set of preference annotations can be collected per user -- a problem we define as Personalized Preference A…

LLM-based Relevance Assessment for Web-Scale Search Evaluation at Pinterest
Han Wang, Alex Whitworth, Pak Ming Cheung, Zhenjie Zhang, Krishna Kamath
https://arxiv.org/abs/2509.03764

LLM-based Relevance Assessment for Web-Scale Search Evaluation at Pinterest
Relevance evaluation plays a crucial role in personalized search systems to ensure that search results align with a user's queries and intent. While human annotation is the traditional method for relevance evaluation, its high cost and long turnaround time limit its scalability. In this work, we present our approach at Pinterest Search to automate relevance evaluation for online experiments using fine-tuned LLMs. We rigorously validate the alignment between LLM-generated judgments and human ann…

Is LLM-Generated Code More Maintainable \& Reliable than Human-Written Code?
Alfred Santa Molison, Marcia Moraes, Glaucia Melo, Fabio Santos, Wesley K. G. Assuncao
https://arxiv.org/abs/2508.00700 …

Is LLM-Generated Code More Maintainable \& Reliable than Human-Written Code?
Background: The rise of Large Language Models (LLMs) in software development has opened new possibilities for code generation. Despite the widespread use of this technology, it remains unclear how well LLMs generate code solutions in terms of software quality and how they compare to human-written code. Aims: This study compares the internal quality attributes of LLM-generated and human-written code. Method: Our empirical study integrates datasets of coding tasks, three LLM configurations (zero-…

FaMA: LLM-Empowered Agentic Assistant for Consumer-to-Consumer Marketplace
Yineng Yan, Xidong Wang, Jin Seng Cheng, Ran Hu, Wentao Guan, Nahid Farahmand, Hengte Lin, Yue Li
https://arxiv.org/abs/2509.03890

FaMA: LLM-Empowered Agentic Assistant for Consumer-to-Consumer Marketplace
The emergence of agentic AI, powered by Large Language Models (LLMs), marks a paradigm shift from reactive generative systems to proactive, goal-oriented autonomous agents capable of sophisticated planning, memory, and tool use. This evolution presents a novel opportunity to address long-standing challenges in complex digital environments. Core tasks on Consumer-to-Consumer (C2C) e-commerce platforms often require users to navigate complex Graphical User Interfaces (GUIs), making the experience…

AutoPBO: LLM-powered Optimization for Local Search PBO Solvers
Jinyuan Li, Yi Chu, Yiwen Sun, Mengchuan Zou, Shaowei Cai
https://arxiv.org/abs/2509.04007 https://

AutoPBO: LLM-powered Optimization for Local Search PBO Solvers
Pseudo-Boolean Optimization (PBO) provides a powerful framework for modeling combinatorial problems through pseudo-Boolean (PB) constraints. Local search solvers have shown excellent performance in PBO solving, and their efficiency is highly dependent on their internal heuristics to guide the search. Still, their design often requires significant expert effort and manual tuning in practice. While Large Language Models (LLMs) have demonstrated potential in automating algorithm design, their appl…

Leveraging LLM-Based Agents for Intelligent Supply Chain Planning
Yongzhi Qi, Jiaheng Yin, Jianshen Zhang, Dongyang Geng, Zhengyu Chen, Hao Hu, Wei Qi, Zuo-Jun Max Shen
https://arxiv.org/abs/2509.03811

Leveraging LLM-Based Agents for Intelligent Supply Chain Planning
In supply chain management, planning is a critical concept. The movement of physical products across different categories, from suppliers to warehouse management, to sales, and logistics transporting them to customers, entails the involvement of many entities. It covers various aspects such as demand forecasting, inventory management, sales operations, and replenishment. How to collect relevant data from an e-commerce platform's perspective, formulate long-term plans, and dynamically adjust the…

Breaking the Mirror: Activation-Based Mitigation of Self-Preference in LLM Evaluators
Dani Roytburg, Matthew Bozoukov, Matthew Nguyen, Jou Barzdukas, Simon Fu, Narmeen Oozeer
https://arxiv.org/abs/2509.03647

Breaking the Mirror: Activation-Based Mitigation of Self-Preference in LLM Evaluators
Large language models (LLMs) increasingly serve as automated evaluators, yet they suffer from "self-preference bias": a tendency to favor their own outputs over those of other models. This bias undermines fairness and reliability in evaluation pipelines, particularly for tasks like preference tuning and model routing. We investigate whether lightweight steering vectors can mitigate this problem at inference time without retraining. We introduce a curated dataset that distinguishes self-preferen…

What Would an LLM Do? Evaluating Policymaking Capabilities of Large Language Models
Pierre Le Coz, Jia An Liu, Debarun Bhattacharjya, Georgina Curto, Serge Stinckwich
https://arxiv.org/abs/2509.03827

What Would an LLM Do? Evaluating Policymaking Capabilities of Large Language Models
Large language models (LLMs) are increasingly being adopted in high-stakes domains. Their capacity to process vast amounts of unstructured data, explore flexible scenarios, and handle a diversity of contextual factors can make them uniquely suited to provide new insights for the complexity of social policymaking. This article evaluates whether LLMs' are aligned with domain experts (and among themselves) to inform social policymaking on the subject of homelessness alleviation - a challenge affec…

Measuring Bias or Measuring the Task: Understanding the Brittle Nature of LLM Gender Biases
Bufan Gao, Elisa Kreiss
https://arxiv.org/abs/2509.04373 https://

Measuring Bias or Measuring the Task: Understanding the Brittle Nature of LLM Gender Biases
As LLMs are increasingly applied in socially impactful settings, concerns about gender bias have prompted growing efforts both to measure and mitigate such bias. These efforts often rely on evaluation tasks that differ from natural language distributions, as they typically involve carefully constructed task prompts that overtly or covertly signal the presence of gender bias-related content. In this paper, we examine how signaling the evaluative purpose of a task impacts measured gender bias in …

Scaling LLM Planning: NL2FLOW for Parametric Problem Generation and Rigorous Evaluation
Jungkoo Kang
https://arxiv.org/abs/2507.02253 https://

Scaling LLM Planning: NL2FLOW for Parametric Problem Generation and Rigorous Evaluation
Progress in enhancing large language model (LLM) planning and reasoning capabilities is significantly hampered by the bottleneck of scalable, reliable data generation and evaluation. To overcome this, I introduce NL2FLOW, a fully automated system for parametrically generating planning problems - expressed in natural language, a structured intermediate representation, and formal PDDL - and rigorously evaluating the quality of generated plans. I demonstrate NL2FLOW's capabilities by generating a …

ArcMemo: Abstract Reasoning Composition with Lifelong LLM Memory
Matthew Ho, Chen Si, Zhaoxiang Feng, Fangxu Yu, Zhijian Liu, Zhiting Hu, Lianhui Qin
https://arxiv.org/abs/2509.04439

ArcMemo: Abstract Reasoning Composition with Lifelong LLM Memory
While inference-time scaling enables LLMs to carry out increasingly long and capable reasoning traces, the patterns and insights uncovered during these traces are immediately discarded once the context window is reset for a new query. External memory is a natural way to persist these discoveries, and recent work has shown clear benefits for reasoning-intensive tasks. We see an opportunity to make such memories more broadly reusable and scalable by moving beyond instance-based memory entries (e.…

Learning When to Plan: Efficiently Allocating Test-Time Compute for LLM Agents
Davide Paglieri, Bart{\l}omiej Cupia{\l}, Jonathan Cook, Ulyana Piterbarg, Jens Tuyls, Edward Grefenstette, Jakob Nicolaus Foerster, Jack Parker-Holder, Tim Rockt\"aschel
https://arxiv.org/abs/2509.03581

Learning When to Plan: Efficiently Allocating Test-Time Compute for LLM Agents
Training large language models (LLMs) to reason via reinforcement learning (RL) significantly improves their problem-solving capabilities. In agentic settings, existing methods like ReAct prompt LLMs to explicitly plan before every action; however, we demonstrate that always planning is computationally expensive and degrades performance on long-horizon tasks, while never planning further limits performance. To address this, we introduce a conceptual framework formalizing dynamic planning for LL…

OMS: On-the-fly, Multi-Objective, Self-Reflective Ad Keyword Generation via LLM Agent
Bowen Chen, Zhao Wang, Shingo Takamatsu
https://arxiv.org/abs/2507.02353

OMS: On-the-fly, Multi-Objective, Self-Reflective Ad Keyword Generation via LLM Agent
Keyword decision in Sponsored Search Advertising is critical to the success of ad campaigns. While LLM-based methods offer automated keyword generation, they face three major limitations: reliance on large-scale query-keyword pair data, lack of online multi-objective performance monitoring and optimization, and weak quality control in keyword selection. These issues hinder the agentic use of LLMs in fully automating keyword decisions by monitoring and reasoning over key performance indicators s…

Efficient Agents: Building Effective Agents While Reducing Cost
Ningning Wang, Xavier Hu, Pai Liu, He Zhu, Yue Hou, Heyuan Huang, Shengyu Zhang, Jian Yang, Jiaheng Liu, Ge Zhang, Changwang Zhang, Jun Wang, Yuchen Eleanor Jiang, Wangchunshu Zhou
https://arxiv.org/abs/2508.02694

Efficient Agents: Building Effective Agents While Reducing Cost
The remarkable capabilities of Large Language Model (LLM)-driven agents have enabled sophisticated systems to tackle complex, multi-step tasks, but their escalating costs threaten scalability and accessibility. This work presents the first systematic study of the efficiency-effectiveness trade-off in modern agent systems, addressing the critical need for cost-effective designs without sacrificing performance. We investigate three key questions: (1) How much complexity do agentic tasks inherentl…

Do Role-Playing Agents Practice What They Preach? Belief-Behavior Consistency in LLM-Based Simulations of Human Trust
Amogh Mannekote, Adam Davies, Guohao Li, Kristy Elizabeth Boyer, ChengXiang Zhai, Bonnie J Dorr, Francesco Pinto
https://arxiv.org/abs/2507.02197

Do Role-Playing Agents Practice What They Preach? Belief-Behavior Consistency in LLM-Based Simulations of Human Trust
As LLMs are increasingly studied as role-playing agents to generate synthetic data for human behavioral research, ensuring that their outputs remain coherent with their assigned roles has become a critical concern. In this paper, we investigate how consistently LLM-based role-playing agents' stated beliefs about the behavior of the people they are asked to role-play ("what they say") correspond to their actual behavior during role-play ("how they act"). Specifically, we establish an evaluation …