Tootfinder

Deconstructing Attention: Investigating Design Principles for Effective Language Modeling
Huiyin Xue, Nafise Sadat Moosavi, Nikolaos Aletras
https://arxiv.org/abs/2510.11602 htt…

Deconstructing Attention: Investigating Design Principles for Effective Language Modeling
The success of Transformer language models is widely credited to their dot-product attention mechanism, which interweaves a set of key design principles: mixing information across positions (enabling multi-token interactions), sequence-dependent activations (where attention weights adapt to each input), a specific mathematical form (dot-product similarities plus softmax weighting), and coupling of queries and keys to evolving hidden states (grounding attention in the current layer). However, th…

wiki_talk: Wikipedia talk networks
Interactions among users of 10 language-specific Wikipedias: Arabic, Chinese, Dutch, English, French, German, Italian, Portuguese, Russian, and Spanish. Nodes are registered wiki editors, and an edge represents a user i having written a message on user j's talk page. Edges are timestamped. The precise dates of the snapshots are uncertain.
This network has 225749 nodes and 1554699 edges.
Tags: Social, Communication, Unweighted, Multigra…

Large Language Models Are Effective Code Watermarkers
Rui Xu, Jiawei Chen, Zhaoxia Yin, Cong Kong, Xinpeng Zhang
https://arxiv.org/abs/2510.11251 https://a…

Large Language Models Are Effective Code Watermarkers
The widespread use of large language models (LLMs) and open-source code has raised ethical and security concerns regarding the distribution and attribution of source code, including unauthorized redistribution, license violations, and misuse of code for malicious purposes. Watermarking has emerged as a promising solution for source attribution, but existing techniques rely heavily on hand-crafted transformation rules, abstract syntax tree (AST) manipulation, or task-specific training, limiting …

Query-Specific GNN: A Comprehensive Graph Representation Learning Method for Retrieval Augmented Generation
Yuchen Yan, Zhihua Liu, Hao Wang, Weiming Li, Xiaoshuai Hao
https://arxiv.org/abs/2510.11541 …

Query-Specific GNN: A Comprehensive Graph Representation Learning Method for Retrieval Augmented Generation
Retrieval-augmented generation (RAG) has demonstrated its ability to enhance Large Language Models (LLMs) by integrating external knowledge sources. However, multi-hop questions, which require the identification of multiple knowledge targets to form a synthesized answer, raise new challenges for RAG systems. Under the multi-hop settings, existing methods often struggle to fully understand the questions with complex semantic structures and are susceptible to irrelevant noise during the retrieval…

Does LLM Focus on the Right Words? Diagnosing Language Bias in LLM-based Recommenders
Bohao Wang, Jiawei Chen, Feng Liu, Changwang Zhang, Jun Wang, Canghong Jin, Chun Chen, Can Wang
https://arxiv.org/abs/2510.10978

Does LLM Focus on the Right Words? Diagnosing Language Bias in LLM-based Recommenders
Large language models (LLMs), owing to their extensive open-domain knowledge and semantic reasoning capabilities, have been increasingly integrated into recommender systems (RS). However, a substantial gap remains between the pre-training objectives of LLMs and the specific requirements of recommendation tasks. To address this gap, supervised fine-tuning (SFT) is commonly performed on specially curated recommendation datasets to further enhance their predictive ability. Despite its success, SFT…

Had a great time presenting our paper “Domain-specific tensor languages” today at #ICFP2025 in Singapore 🇸🇬
The images attached are screenshots from the paper as teasers — diagrams for derivative rules, the Riemann curvature 4-tensor in Einstein’s index notation, and the same in diagram form.
📄 Slides:

ALLOY: Generating Reusable Agent Workflows from User Demonstration
Jiawen Li, Zheng Ning, Yuan Tian, Toby Jia-jun Li
https://arxiv.org/abs/2510.10049 https://

ALLOY: Generating Reusable Agent Workflows from User Demonstration
Large language models (LLMs) enable end-users to delegate complex tasks to autonomous agents through natural language. However, prompt-based interaction faces critical limitations: Users often struggle to specify procedural requirements for tasks, especially those that don't have a factually correct solution but instead rely on personal preferences, such as posting social media content or planning a trip. Additionally, a ''successful'' prompt for one task may not be reusable or generalizable ac…

MeTA-LoRA: Data-Efficient Multi-Task Fine-Tuning for Large Language Models
Bo Cheng, Xu Wang, Jinda Liu, Yi Chang, Yuan Wu
https://arxiv.org/abs/2510.11598 https://

MeTA-LoRA: Data-Efficient Multi-Task Fine-Tuning for Large Language Models
Low-Rank Adaptation (LoRA) has emerged as one of the most widely used parameter-efficient fine-tuning (PEFT) methods for adapting large language models (LLMs) to downstream tasks. While highly effective in single-task settings, it struggles to efficiently leverage inter-task knowledge in complex multi-task learning scenarios, often requiring substantial task-specific data to achieve optimal performance. To address this limitation, we introduce MeTA-LoRA, a two-stage optimization framework that …

Language conversations have become highly performative,
with a fixation on the words people are using,
not why they’re using them
or what they’re reaching for when they deploy language
like “crazy,” “lame,” or the r-word
(currently experiencing a resurgence in popularitythanks to Elon Musk).
The language is a metaphor:
We call something “insane” because that word carries a specific baggage and burden.
But the fact that mental illness is stil…

Language Wars
Conversations about language were supposed to be a starting point rather than an ending one.

LLM-Specific Utility: A New Perspective for Retrieval-Augmented Generation
Hengran Zhang, Keping Bi, Jiafeng Guo, Jiaming Zhang, Shuaiqiang Wang, Dawei Yin, Xueqi Cheng
https://arxiv.org/abs/2510.11358

LLM-Specific Utility: A New Perspective for Retrieval-Augmented Generation
Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating external knowledge. While traditional retrieval focuses on relevance, RAG's effectiveness depends on the utility of retrieved passages, i.e., the usefulness in facilitating the generation of an accurate and comprehensive answer. Existing studies often treat utility as a generic attribute, ignoring the fact that different LLMs may benefit differently from the same passage due to variations in internal kno…