Tootfinder

MeAJOR Corpus: A Multi-Source Dataset for Phishing Email Detection
Paulo Mendes (GECAD, ISEP, Polytechnic of Porto, Portugal), Eva Maia (GECAD, ISEP, Polytechnic of Porto, Portugal), Isabel Pra\c{c}a (GECAD, ISEP, Polytechnic of Porto, Portugal)
https://arxiv.org/abs/2507.17978

MeAJOR Corpus: A Multi-Source Dataset for Phishing Email Detection
Phishing emails continue to pose a significant threat to cybersecurity by exploiting human vulnerabilities through deceptive content and malicious payloads. While Machine Learning (ML) models are effective at detecting phishing threats, their performance largely relies on the quality and diversity of the training data. This paper presents MeAJOR (Merged email Assets from Joint Open-source Repositories) Corpus, a novel, multi-source phishing email dataset designed to overcome critical limitation…

HARPT: A Corpus for Analyzing Consumers' Trust and Privacy Concerns in Mobile Health Apps
Timoteo Kelly, Abdulkadir Korkmaz, Samuel Mallet, Connor Souders, Sadra Aliakbarpour, Praveen Rao
https://arxiv.org/abs/2506.19268

HARPT: A Corpus for Analyzing Consumers' Trust and Privacy Concerns in Mobile Health Apps
We present HARPT, a large-scale annotated corpus of mobile health app store reviews aimed at advancing research in user privacy and trust. The dataset comprises over 480,000 user reviews labeled into seven categories that capture critical aspects of trust in applications, trust in providers and privacy concerns. Creating HARPT required addressing multiple complexities, such as defining a nuanced label schema, isolating relevant content from large volumes of noisy data, and designing an annotati…

JIS: A Speech Corpus of Japanese Idol Speakers with Various Speaking Styles
Yuto Kondo, Hirokazu Kameoka, Kou Tanaka, Takuhiro Kaneko
https://arxiv.org/abs/2506.18296

JIS: A Speech Corpus of Japanese Idol Speakers with Various Speaking Styles
We construct Japanese Idol Speech Corpus (JIS) to advance research in speech generation AI, including text-to-speech synthesis (TTS) and voice conversion (VC). JIS will facilitate more rigorous evaluations of speaker similarity in TTS and VC systems since all speakers in JIS belong to a highly specific category: "young female live idols" in Japan, and each speaker is identified by a stage name, enabling researchers to recruit listeners familiar with these idols for listening experiments. With i…

Harnessing the Power of Reinforcement Learning for Language-Model-Based Information Retriever via Query-Document Co-Augmentation
Jingming Liu, Yumeng Li, Wei Shi, Yao-Xiang Ding, Hui Su, Kun Zhou
https://arxiv.org/abs/2506.18670

Harnessing the Power of Reinforcement Learning for Language-Model-Based Information Retriever via Query-Document Co-Augmentation
Recent studies have proposed leveraging Large Language Models (LLMs) as information retrievers through query rewriting. However, for challenging corpora, we argue that enhancing queries alone is insufficient for robust semantic matching; the LLM should also have sufficient understanding of the corpus by directly handling and augmenting the documents themselves. To this end, we present an LLM-based retriever empowered to augment both user queries and corpus documents, with its policy fully explo…

HathiTrust was founded in 2008 as a not-for-profit collaborative of academic and research libraries now preserving 18 million digitized items in the HathiTrust Digital Library. We offer reading access to the fullest extent allowable by U.S. and international copyright law, text and data mining tools for the entire corpus, and other emerging services based on the combined collection.

email_enron: Email network (Enron corpus)
The Enron email corpus, containing all the email communication from the Enron corporation, which was made public as a result of legal action. Nodes are email addresses and node i links to node j if i sent at least one email to address j. Non-Enron email addresses are also present, but only their links to/from Enron addresses are observed.
This network has 36692 nodes and 367662 edges.
Tags: Social, Communication, Unweighted, Multigr…

“‘It’s obvious that you don’t respect Copyright Law and Artist Rights any more than you respect Habeas Corpus and Due Process rights, not to mention the separation of Church and State per the US Constitution. For the record, we hereby order dhsgov [US Department of Homeland Security] to cease and desist the use of our recording and demand that you immediately pull down your video.’”
“They added: ‘Oh, and go f… yourselves.’”

Black Rebel Motorcycle Club issue cease and desist to US Homeland Security over "improper" use of song in "propaganda video"
Black Rebel Motorcycle Club have issued a cease-and-desist notice to US Homeland Security over their use of the band’s music. 

At least people across academic disciplines are selling themselves out to "AI" bullshit.
Here: Let's get some software make wrong autocompletions of Latin texts and call it scholarship, while probably poisoning our knowledge corpus for years to come…
https://archive.ph/U0ePh

"White House deputy chief of staff Stephen Miller says President Donald Trump is looking for ways to expand its legal power to deport migrants who are in the United States illegally. To achieve that, he says the administration is “actively looking at” suspending habeas corpus, . ."

Trump team mulls suspending habeas corpus to speed deportations. Can it?
Top White House aide Stephen Miller says the Trump administration is looking for ways to expand its legal power to deport migrants who are in the United States illegally. To achieve that, he says President Donald Trump’s team is “actively looking at” suspending habeas corpus, the constitutional right for people to legally challenge their detention by the government. Such a move would be aimed at migrants as part of the Republican president’s broader crackdown at the U.S.-Mexico border. …

- We investigate the distribution of the feature variables (corpus frequency, dictionary views, part-of-speech, polysemy) over CEFR levels.
- Variable importance analyses show us how important each variable was for the classification of each level.
We conclude: "Thus, our semi-automatic approach offers a practical solution to the limitations of existing CEFR lists, providing a framework for expanding these lists in a systematic and data-driven manner. However, our findings also reveal the importance of human oversight in the process."
Supplementary material contains an ensemble approach to classification and all used/generated data: https://osf.io/6s9y7/

Identifying Pre-training Data in LLMs: A Neuron Activation-Based Detection Framework
Hongyi Tang, Zhihao Zhu, Yi Yang
https://arxiv.org/abs/2507.16414 http…

Identifying Pre-training Data in LLMs: A Neuron Activation-Based Detection Framework
The performance of large language models (LLMs) is closely tied to their training data, which can include copyrighted material or private information, raising legal and ethical concerns. Additionally, LLMs face criticism for dataset contamination and internalizing biases. To address these issues, the Pre-Training Data Detection (PDD) task was proposed to identify if specific data was included in an LLM's pre-training corpus. However, existing PDD methods often rely on superficial features like …

Never Come Up Empty: Adaptive HyDE Retrieval for Improving LLM Developer Support
Fangjian Lei, Mariam El Mezouar, Shayan Noei, Ying Zou
https://arxiv.org/abs/2507.16754

Never Come Up Empty: Adaptive HyDE Retrieval for Improving LLM Developer Support
Large Language Models (LLMs) have shown promise in assisting developers with code-related questions; however, LLMs carry the risk of generating unreliable answers. To address this, Retrieval-Augmented Generation (RAG) has been proposed to reduce the unreliability (i.e., hallucinations) of LLMs. However, designing effective pipelines remains challenging due to numerous design choices. In this paper, we construct a retrieval corpus of over 3 million Java and Python related Stack Overflow posts wi…

Join Radu Pop and Pietro Mele at Berlin Buzzwords as they discuss building an extensible hybrid search solution with Elasticsearch. They will cover functional modeling, cluster architecture, and practical insights on managing billions of vectors in real-world scenarios. Radu and Pietro will also address hybrid reranking challenges and the limitations of standard fusion techniques, explaining their innovative approach.
Learn more:

The U.S. Department of Housing and Urban Development is preparing to
💥shut down seven major investigations and cases concerning alleged housing discrimination and segregation, including
⚠️ some where the agency already found civil rights violations, according to HUD records obtained by ProPublica.
The high-profile cases involve allegations that state and local governments across the South and Midwest illegally discriminated against people of color
by placing industrial…

Trump Administration Prepares to Drop Seven Major Housing Discrimination Cases
Federal housing officials spent years investigating cities from Chicago to Memphis to Corpus Christi for putting industrial plants and unwanted facilities in poor, nonwhite neighborhoods. Now, under Trump, the agency plans to drop the cases.

Disaster Informatics after the COVID-19 Pandemic: Bibliometric and Topic Analysis based on Large-scale Academic Literature
Ngan Tran, Haihua Chen, Ana Cleveland, Yuhan Zhou
https://arxiv.org/abs/2507.16820

Disaster Informatics after the COVID-19 Pandemic: Bibliometric and Topic Analysis based on Large-scale Academic Literature
This study presents a comprehensive bibliometric and topic analysis of the disaster informatics literature published between January 2020 to September 2022. Leveraging a large-scale corpus and advanced techniques such as pre-trained language models and generative AI, we identify the most active countries, institutions, authors, collaboration networks, emergent topics, patterns among the most significant topics, and shifts in research priorities spurred by the COVID-19 pandemic. Our findings hig…

Since Kristi Noem doesn't appear to know what habeas corpus is, I'm updating my fantasy version of the Constitution (where I imagine how I might edit it) to spell it out a bit more explicitly:
https://github.com/cooljeanius/w_Constitution/compare/f0deb279b70e2768ff16e53ceb00896b82f67259...ca1304a3c0ecb78f50f263d5bfedda0f3a08b78e

Pr\"{u}fer codes on vertex-colored rooted trees
R. W. R. Darling, Grant Fickes
https://arxiv.org/abs/2506.15796 https://arxiv.or…

Prüfer codes on vertex-colored rooted trees
Prüfer codes provide an encoding scheme for representing a vertex-labeled tree on $n$ vertices with a string of length $n-2$. Indeed, two labeled trees are isomorphic if and only if their Prüfer codes are identical, and this supplies a proof of Cayley's Theorem. Motivated by a graph decomposition of freight networks into a corpus of vertex-colored rooted trees, we extend the notion of Prüfer codes to that setting, i.e., trees without a unique labeling, by defining a canonical label for a ver…

#DH2025 thanks @flochiff.bsky.social for sharing this link as I wanted to follow up on Pandore! 'Pandore: automating text-processing workflows for humanities researchers' from Sorbonne Université and ObTIC - Observatoire des textes, des idées et des corpus

The Expressions of Depression and Anxiety in Chinese Psycho-counseling: Usage of First-person Singular Pronoun and Negative Emotional Words
Lizhi Ma, Tong Zhao, Shuai Zhang, Nirui Song, Hongliang He, Anqi Li, Ran Feng, Huachuan Qiu, Jingsong Ma, Zhenzhong Lan
https://arxiv.org/abs/2507.13839…

The Expressions of Depression and Anxiety in Chinese Psycho-counseling: Usage of First-person Singular Pronoun and Negative Emotional Words
This study explores the relationship between linguistic expressions and psychological states of depression and anxiety within Chinese psycho-counseling interactions, focusing specifically on the usage of first-person singular pronouns and negative emotional words. Utilizing a corpus derived from 735 online counseling sessions, the analysis employed a general linear mixed-effect model to assess linguistic patterns quantified by the Linguistic Inquiry and Word Count (LIWC) software. Results indic…

A new XML conversion process for mensural music encoding : CMME\_to\_MEI (via Verovio)
David Fiala (CESR), Laurent Pugin (KNAW), Marnix van Berchum (KNAW), Martha Thomae (NOVA), K\'evin Roger (CESR, UL, CRULH)
https://arxiv.org/abs/2507.15991

A new XML conversion process for mensural music encoding : CMME\_to\_MEI (via Verovio)
The Ricercar Lab - the musicological research team at the Center for advanced Studies in the Renaissance at the University of Tours - has decided to make available in open access, thanks to the support of the French digital infrastructure Biblissima, a large corpus of about 3500 XML files of 15th-c. music. This corpus was produced by the German musicologist Clemens Goldberg who encoded since 2010 onwards the musical content of 34 major 15th-c. music manuscripts and other complementary files, in…

Corpus Christie
#SłowaNaOpak

#Drumpf #DOJ suing all 15 judges on the #Maryland #federal court bench over imposition of mandatory 2 day delay policy in any

Trump DHS sues entire bench of federal judges in Maryland district court over automatic injunctions
The Trump administration files lawsuit against all 15 Maryland federal judges, challenging court policy that temporarily blocks deportations of immigrants filing habeas corpus petitions.

Neural Cellular Automata for ARC-AGI
Kevin Xu, Risto Miikkulainen
https://arxiv.org/abs/2506.15746 https://arxiv.org/pdf/2506.15746…

Neural Cellular Automata for ARC-AGI
Cellular automata and their differentiable counterparts, Neural Cellular Automata (NCA), are highly expressive and capable of surprisingly complex behaviors. This paper explores how NCAs perform when applied to tasks requiring precise transformations and few-shot generalization, using the Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI) as a domain that challenges their capabilities in ways not previously explored. Specifically, this paper uses gradient-based train…

The Ambient Hermit – Lauda Sion (Reinterpreted)
#ambient

Over the past twenty-one years, I've posted 365 posts to my blog, on average one every twenty-one days. They total almost half a million words.
That's quite some corpus of work.
#Blog
#Blogging

365 posts
Checking my blog yesterday, I noticed that I had published 365 posts, one for every day of the year. Which, considering that the oldest post that remains is from the 24th June 2004, almost twenty one years ago, means that I've averaged one post every twenty-one days over that period. Mind you, I've not been consistent; there've been periods I've posted more, others I've posted less.

From Fiber Tracts to Tumor Spread: Biophysical Modeling of Butterfly Glioma Growth Using Diffusion Tensor Imaging
Jonas Weidner, Ivan Ezhov, Michal Balcerak, Andr\'e Datchev, Lucas Zimmer, Daniel Rueckert, Bj\"orn Menze, Benedikt Wiestler
https://arxiv.org/abs/2507.17707

From Fiber Tracts to Tumor Spread: Biophysical Modeling of Butterfly Glioma Growth Using Diffusion Tensor Imaging
Butterfly tumors are a distinct class of gliomas that span the corpus callosum, producing a characteristic butterfly-shaped appearance on MRI. The distinctive growth pattern of these tumors highlights how white matter fibers and structural connectivity influence brain tumor cell migration. To investigate this relation, we applied biophysical tumor growth models to a large patient cohort, systematically comparing models that incorporate fiber tract information with those that do not. Our results…

An 8TB corpus of copyright-free text for training AI models. Comes with two 7B models trained as proof-of-concept.
https://github.com/r-three/common-pile/blob/main/paper.pdf
I'm glad somebody has finally done this. The "we need to break copyrig…

email_enron: Email network (Enron corpus)
The Enron email corpus, containing all the email communication from the Enron corporation, which was made public as a result of legal action. Nodes are email addresses and node i links to node j if i sent at least one email to address j. Non-Enron email addresses are also present, but only their links to/from Enron addresses are observed.
This network has 36692 nodes and 367662 edges.
Tags: Social, Communication, Unweighted, Multigr…

OH in corpus-linguistic UX convo:
> He who go too far down long tail end up wagging dog

Last week, our students learned how to conduct a proper evaluation for an NLP experiment. To this end, we introduced a small textcorpus with sentences about Joseph Fourier, who counts as one of the discoverers of the greenhouse effect, responsible for global warming.

From Generality to Mastery: Composer-Style Symbolic Music Generation via Large-Scale Pre-training
Mingyang Yao, Ke Chen
https://arxiv.org/abs/2506.17497 ht…

From Generality to Mastery: Composer-Style Symbolic Music Generation via Large-Scale Pre-training
Despite progress in controllable symbolic music generation, data scarcity remains a challenge for certain control modalities. Composer-style music generation is a prime example, as only a few pieces per composer are available, limiting the modeling of both styles and fundamental music elements (e.g., melody, chord, rhythm). In this paper, we investigate how general music knowledge learned from a broad corpus can enhance the mastery of specific composer styles, with a focus on piano piece genera…

Warframe shows us a world full of bosses and big corporations crushing the little people. The Corpus exploit workers, the Grineer enforce control with brute force, and the Solaris fight back to survive.
It’s cool to see the game hint at worker solidarity and rebellion. But at the same time, the game makes me grind endlessly or pay up real cash, which feels a lot like the very system it tries to criticize.
Warframe tells a story about breaking chains, but still keeps me locked in …

Adapting Whisper for Lightweight and Efficient Automatic Speech Recognition of Children for On-device Edge Applications
Satwik Dutta, Shruthigna Chandupatla, John Hansen
https://arxiv.org/abs/2507.14451

Adapting Whisper for Lightweight and Efficient Automatic Speech Recognition of Children for On-device Edge Applications
Reliability on cloud providers for ASR inference to support child-centered voice-based applications is becoming challenging due to regulatory and privacy challenges. Motivated by a privacy-preserving design, this study aims to develop a lightweight & efficient Whisper ASR system capable of running on a Raspberry Pi. Upon evaluation of the MyST corpus and by examining various filtering strategies to fine-tune the `tiny.en' model, a Word Error Rate (WER) of 15.9% was achieved (11.8% filtered). A …

AKEGEN: A LLM-based Tabular Corpus Generator for Evaluating Dataset Discovery in Data Lakes
Zhenwei Dai, Chuan Lei, Asterios Katsifodimos, Xiao Qin, Christos Faloutsos, Huzefa Rangwala
https://arxiv.org/abs/2507.04687

AKEGEN: A LLM-based Tabular Corpus Generator for Evaluating Dataset Discovery in Data Lakes
How to generate a large, realistic set of tables along with joinability relationships, to stress-test dataset discovery methods? Dataset discovery methods aim to automatically identify related data assets in a data lake. The development and evaluation of such solutions for customers from a wide range of business domains, relies on diverse, high quality and domain-specific tabular benchmarks. Large language models (LLMs) are trained on a wide variety of text data, which can provide a strong foun…

MDC-R: The Minecraft Dialogue Corpus with Reference
Chris Madge, Maris Camilleri, Paloma Carretero Garcia, Mladen Karan, Juexi Shao, Prashant Jayannavar, Julian Hough, Benjamin Roth, Massimo Poesio
https://arxiv.org/abs/2506.22062

MDC-R: The Minecraft Dialogue Corpus with Reference
We introduce the Minecraft Dialogue Corpus with Reference (MDC-R). MDC-R is a new language resource that supplements the original Minecraft Dialogue Corpus (MDC) with expert annotations of anaphoric and deictic reference. MDC's task-orientated, multi-turn, situated dialogue in a dynamic environment has motivated multiple annotation efforts, owing to the interesting linguistic phenomena that this setting gives rise to. We believe it can serve as a valuable resource when annotated with reference,…

Extracting Document Relations from Search Corpus by Marginalizing over User Queries
Yuki Iwamoto, Kaoru Tsunoda, Ken Kaneiwa
https://arxiv.org/abs/2507.10726

Extracting Document Relations from Search Corpus by Marginalizing over User Queries
Understanding relationships between documents in large-scale corpora is essential for knowledge discovery and information organization. However, existing approaches rely heavily on manual annotation or predefined relationship taxonomies. We propose EDR-MQ (Extracting Document Relations by Marginalizing over User Queries), a novel framework that discovers document relationships through query marginalization. EDR-MQ is based on the insight that strongly related documents often co-occur in results…

I’ve been eying @… for some time, but I haven’t had time to play with it yet. In case you never heard about OpenRewrite, OpenRewrite takes care of refactoring your codebase to newer language, framework, and paradigm versions.
Using OpenRewrite is pretty straightforward. It already provides a large corpus of existing recipes, some of which are free.…

Best Practices and Considerations for Child Speech Corpus Collection and Curation in Educational, Clinical, and Forensic Scenarios
John Hansen, Satwik Dutta, Ellen Grand
https://arxiv.org/abs/2507.12870

Best Practices and Considerations for Child Speech Corpus Collection and Curation in Educational, Clinical, and Forensic Scenarios
A child's spoken ability continues to change until their adult age. Until 7-8yrs, their speech sound development and language structure evolve rapidly. This dynamic shift in their spoken communication skills and data privacy make it challenging to curate technology-ready speech corpora for children. This study aims to bridge this gap and provide researchers and practitioners with the best practices and considerations for developing such a corpus based on an intended goal. Although primarily foc…

email_enron: Email network (Enron corpus)
The Enron email corpus, containing all the email communication from the Enron corporation, which was made public as a result of legal action. Nodes are email addresses and node i links to node j if i sent at least one email to address j. Non-Enron email addresses are also present, but only their links to/from Enron addresses are observed.
This network has 36692 nodes and 367662 edges.
Tags: Social, Communication, Unweighted, Multigr…

Don't throw the baby out with the bathwater: How and why deep learning for ARC
Jack Cole, Mohamed Osman
https://arxiv.org/abs/2506.14276 https://

Don't throw the baby out with the bathwater: How and why deep learning for ARC
The Abstraction and Reasoning Corpus (ARC-AGI) presents a formidable challenge for AI systems. Despite the typically low performance on ARC, the deep learning paradigm remains the most effective known strategy for generating skillful (state-of-the-art) neural networks (NN) across varied modalities and tasks in vision, language etc. The deep learning paradigm has proven to be able to train these skillful neural networks and learn the abstractions needed in these diverse domains. Our work doubles…

🇺🇦 #NowPlaying on BBCRadio3's #MusicMap
Wolfgang Amadeus Mozart, Víkingur œlafsson & Franz Liszt:
🎵 Ave verum corpus
#WolfgangAmadeusMozart #VíkingurÓlafsson #FranzLiszt
https://open.spotify.com/track/6HPnobfvxQk0J56gx1OgB3

Generalist Models in Medical Image Segmentation: A Survey and Performance Comparison with Task-Specific Approaches
Andrea Moglia (Politecnico di Milano), Matteo Leccardi (Politecnico di Milano), Matteo Cavicchioli (Politecnico di Milano), Alice Maccarini (Universit\`a di Pavia), Marco Marcon (Politecnico di Milano), Luca Mainardi (Politecnico di Milano), Pietro Cerveri (Politecnico di Milano, Universit\`a di Pavia)

Generalist Models in Medical Image Segmentation: A Survey and Performance Comparison with Task-Specific Approaches
Following the successful paradigm shift of large language models, leveraging pre-training on a massive corpus of data and fine-tuning on different downstream tasks, generalist models have made their foray into computer vision. The introduction of Segment Anything Model (SAM) set a milestone on segmentation of natural images, inspiring the design of a multitude of architectures for medical image segmentation. In this survey we offer a comprehensive and in-depth investigation on generalist models…

Adopted, because no alttext:
#Ukraine

Even outside the digital transformation of "planning", I'm very interested (as a member of the public) in the development of a rich corpus of public geospatial data. (Some deep unexpressed cartographic instinct?) This blogpost is an update on the status of that:
https://…

Standardising the planning application process: Progress and next steps – MHCLG Digital
The blog of the digital and data teams in MHCLG

from my link log —
10 years of pomological watercolors.
https://parkerhiggins.net/2025/04/10-years-of-pomological-watercolors/
saved 2025-04-13

10 years of pomological watercolors
A decade ago today I published a blog post calling for the US government to release its paintings of fruits. The Pomological Watercolor Collection, as I had recently come to know, is a beautiful and remarkable corpus of over 7,000 pictures of fruits and other biological specimens, made between the 1880s and 1940s. Through a handful of FOIA requests I’d learned that the images had been meticulously digitized and put online for purchase, but that less than 100 pictures had been sold that way �…

SCoRE: Streamlined Corpus-based Relation Extraction using Multi-Label Contrastive Learning and Bayesian kNN
Luca Mariotti, Veronica Guidetti, Federica Mandreoli
https://arxiv.org/abs/2507.06895

SCoRE: Streamlined Corpus-based Relation Extraction using Multi-Label Contrastive Learning and Bayesian kNN
The growing demand for efficient knowledge graph (KG) enrichment leveraging external corpora has intensified interest in relation extraction (RE), particularly under low-supervision settings. To address the need for adaptable and noise-resilient RE solutions that integrate seamlessly with pre-trained large language models (PLMs), we introduce SCoRE, a modular and cost-effective sentence-level RE system. SCoRE enables easy PLM switching, requires no finetuning, and adapts smoothly to diverse cor…

The sheer confidence in her false answer defines insanity. It's unbelievable how resilient the US separation of powers is ->
Kristi Noem Falsely Defines Habeas Corpus as Trump’s Right to Deport People | The New York Times
https://www.youtube.com/shorts/aNf3Yb5N4J4

TTSOps: A Closed-Loop Corpus Optimization Framework for Training Multi-Speaker TTS Models from Dark Data
Kentaro Seki, Shinnosuke Takamichi, Takaaki Saeki, Hiroshi Saruwatari
https://arxiv.org/abs/2506.15614

TTSOps: A Closed-Loop Corpus Optimization Framework for Training Multi-Speaker TTS Models from Dark Data
This paper presents TTSOps, a fully automated closed-loop framework for constructing multi-speaker text-to-speech (TTS) systems from noisy, uncurated web-scale speech data, often referred to as ``dark data,'' such as online videos. Conventional TTS training pipelines require well-curated corpora with high acoustic quality and accurate text-speech alignment, which severely limits scalability, speaker diversity, and real-world applicability. While recent studies have proposed acoustic-quality-bas…

Towards LLM-based Root Cause Analysis of Hardware Design Failures
Siyu Qiu, Muzhi Wang, Raheel Afsharmazayejani, Mohammad Moradi Shahmiri, Benjamin Tan, Hammond Pearce
https://arxiv.org/abs/2507.06512

Towards LLM-based Root Cause Analysis of Hardware Design Failures
With advances in large language models (LLMs), new opportunities have emerged to develop tools that support the digital hardware design process. In this work, we explore how LLMs can assist with explaining the root cause of design issues and bugs that are revealed during synthesis and simulation, a necessary milestone on the pathway towards widespread use of LLMs in the hardware design process and for hardware security analysis. We find promising results: for our corpus of 34 different buggy sc…

IdolSongsJp Corpus: A Multi-Singer Song Corpus in the Style of Japanese Idol Groups
Hitoshi Suda, Junya Koguchi, Shunsuke Yoshida, Tomohiko Nakamura, Satoru Fukayama, Jun Ogata
https://arxiv.org/abs/2507.01349

IdolSongsJp Corpus: A Multi-Singer Song Corpus in the Style of Japanese Idol Groups
Japanese idol groups, comprising performers known as "idols," are an indispensable part of Japanese pop culture. They frequently appear in live concerts and television programs, entertaining audiences with their singing and dancing. Similar to other J-pop songs, idol group music covers a wide range of styles, with various types of chord progressions and instrumental arrangements. These tracks often feature numerous instruments and employ complex mastering techniques, resulting in high signal lo…

HoneySat: A Network-based Satellite Honeypot Framework
Efr\'en L\'opez-Morales (Texas A&M University-Corpus Christi), Ulysse Planta (CISPA Helmholtz Center for Information Security), Gabriele Marra (CISPA Helmholtz Center for Information Security), Carlos Gonz\'alez (German Aerospace Center), Jacob Hopkins (Texas A&M University-Corpus Christi), Majid Garoosi (CISPA Helmholtz Center for Information Security), El\'ias Obreque (Universidad de Chile), Carlos Rubio-M…

HoneySat: A Network-based Satellite Honeypot Framework
Satellites are the backbone of several mission-critical services, such as GPS that enable our modern society to function. For many years, satellites were assumed to be secure because of their indecipherable architectures and the reliance on security by obscurity. However, technological advancements have made these assumptions obsolete, paving the way for potential attacks, and sparking a renewed interest in satellite security. Unfortunately, to this day, there is no efficient way to collect dat…

Oldies but Goldies: The Potential of Character N-grams for Romanian Texts
Dana Lupsa, Sanda-Maria Avram
https://arxiv.org/abs/2506.15650 https://

Oldies but Goldies: The Potential of Character N-grams for Romanian Texts
This study addresses the problem of authorship attribution for Romanian texts using the ROST corpus, a standard benchmark in the field. We systematically evaluate six machine learning techniques: Support Vector Machine (SVM), Logistic Regression (LR), k-Nearest Neighbors (k-NN), Decision Trees (DT), Random Forests (RF), and Artificial Neural Networks (ANN), employing character n-gram features for classification. Among these, the ANN model achieved the highest performance, including perfect clas…

Just published:
Supplementing CEFR-graded vocabulary lists for language learners by leveraging information on dictionary views, corpus frequency, part-of-speech, and polysemy
A machine-learning method to suggest word candidates for CEFR-graded vocabulary lists.
#CEFR level of previously unlabeled words
#linguistics #CEFR #frequency #dictionary #LanguageLearning

It's interesting how much Wikipedia's list of LLM catchphrases comes straight out of academic writing jargon.
Probably as a lot of the corpus that was mined for the LLM training goes back to the academic literature thanks to Sci-Hub et al.
https://en.wikipedia.org/wiki/Wikipedia:WikiProject_AI_Cleanup/AI_catchphrases#Language_and_tone

AI, AGI, and learning efficiency
My 4-month-old kid is not DDoSing Wikipedia right now, nor will they ever do so before learning to speak, read, or write. Their entire "training corpus" will not top even 100 million "tokens" before they can speak & understand language, and do so with real intentionally.
Just to emphasize that point: 100 words-per-minute times 60 minutes-per-hour times 12 hours-per-day times 365 days-per-year times 4 years is a mere 105,120,000 words. That's a ludicrously *high* estimate of words-per-minute and hours-per-day, and 4 years old (the age of my other kid) is well after basic speech capabilities are developed in many children, etc. More likely the available "training data" is at least 1 or 2 orders of magnitude less than this.
The point here is that large language models, trained as they are on multiple *billions* of tokens, are not developing their behavioral capabilities in a way that's remotely similar to humans, even if you believe those capabilities are similar (they are by certain very biased ways of measurement; they very much aren't by others). This idea that humans must be naturally good at acquiring language is an old one (see e.g. #AI #LLM #AGI

Grammatical Structure and Grammatical Variations in Non-Metric Iranian Classical Music
Maziar Kanani, Sean O Leary, James McDermott
https://arxiv.org/abs/2507.10708

Grammatical Structure and Grammatical Variations in Non-Metric Iranian Classical Music
In this study we introduce a symbolic dataset composed of non-metric Iranian classical music, and algorithms for structural parsing of this music, and generation of variations. The corpus comprises MIDI files and data sheets of Dastgah Shour from Radif Mirza Abdollah, the foundational repertoire of Iranian classical music. Furthermore, we apply our previously-introduced algorithm for parsing melodic structure (Kanani et al., 2023b)to the dataset. Unlike much Western music, this type of non-metr…

Labelling Data with Unknown References
Adrian de Wynter
https://arxiv.org/abs/2506.03083 https://arxiv.org/pdf/2506.03083

Labelling Data with Unknown References
An evaluator is trustworthy when there exists some agreed-upon way to measure its performance as a labeller. The two ways to establish trustworthiness are either by testing it, or by assuming the evaluator `knows' somehow the way to label the corpus. However, if labelled references (e.g., a development set) are unavailable, neither of these approaches work: the former requires the data, and the latter is an assumption, not evidence. To address this, we introduce an algorithm (the `No-Data Algor…

XGraphRAG: Interactive Visual Analysis for Graph-based Retrieval-Augmented Generation
Ke Wang, Bo Pan, Yingchaojie Feng, Yuwei Wu, Jieyi Chen, Minfeng Zhu, Wei Chen
https://arxiv.org/abs/2506.13782

XGraphRAG: Interactive Visual Analysis for Graph-based Retrieval-Augmented Generation
Graph-based Retrieval-Augmented Generation (RAG) has shown great capability in enhancing Large Language Model (LLM)'s answer with an external knowledge base. Compared to traditional RAG, it introduces a graph as an intermediate representation to capture better structured relational knowledge in the corpus, elevating the precision and comprehensiveness of generation results. However, developers usually face challenges in analyzing the effectiveness of GraphRAG on their dataset due to GraphRAG's …

Common Corpus: The Largest Collection of Ethical Data for LLM Pre-Training
Pierre-Carl Langlais, Carlos Rosas Hinostroza, Mattia Nee, Catherine Arnett, Pavel Chizhov, Eliot Krzystof Jones, Ir\`ene Girard, David Mach, Anastasia Stasenko, Ivan P. Yamshchikov
https://arxiv.org/abs/2506.01732

Common Corpus: The Largest Collection of Ethical Data for LLM Pre-Training
Large Language Models (LLMs) are pre-trained on large amounts of data from different sources and domains. These data most often contain trillions of tokens with large portions of copyrighted or proprietary content, which hinders the usage of such models under AI legislation. This raises the need for truly open pre-training data that is compliant with the data security regulations. In this paper, we introduce Common Corpus, the largest open dataset for language model pre-training. The data assem…

This https://arxiv.org/abs/2505.08052 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSI_…

NAZM: Network Analysis of Zonal Metrics in Persian Poetic Tradition
This study formalizes a computational model to simulate classical Persian poets' dynamics of influence through constructing a multi-dimensional similarity network. Using a rigorously curated dataset based on Ganjoor's corpus, we draw upon semantic, lexical, stylistic, thematic, and metrical features to demarcate each poet's corpus. Each is contained within weighted similarity matrices, which are then appended to generate an aggregate graph showing poet-to-poet influence. Further network investi…

This https://arxiv.org/abs/2409.17275 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCR_…

On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains
Retrieval-Augmented Generation (RAG) has been empirically shown to enhance the performance of large language models (LLMs) in knowledge-intensive domains such as healthcare, finance, and legal contexts. Given a query, RAG retrieves relevant documents from a corpus and integrates them into the LLMs' generation process. In this study, we investigate the adversarial robustness of RAG, focusing specifically on examining the retrieval system. First, across 225 different setup combinations of corpus,…

Approximating Language Model Training Data from Weights
John X. Morris, Junjie Oscar Yin, Woojeong Kim, Vitaly Shmatikov, Alexander M. Rush
https://arxiv.org/abs/2506.15553

Approximating Language Model Training Data from Weights
Modern language models often have open weights but closed training data. We formalize the problem of data approximation from model weights and propose several baselines and metrics. We develop a gradient-based approach that selects the highest-matching data from a large public text corpus and show its effectiveness at recovering useful data given only weights of the original and finetuned models. Even when none of the true training data is known, our method is able to locate a small subset of p…

Potential Effects of Loading Terminal Locations on Surface Trajectories of Oil Spill Transport
Shoshana Reich, Edward Buskey, Clint Dawson, Eirik Valseth
https://arxiv.org/abs/2505.24610

Potential Effects of Loading Terminal Locations on Surface Trajectories of Oil Spill Transport
We present an investigation comparing the potential impacts of offshore and onshore crude oil loading sites on surface trajectories of spilled oil particles in the regions near the Port of Corpus Christi, Texas. Oil transport is established in a two step procedure. First, the circulation and flow characteristics of seawater throughout the coastal ocean are established for various flow conditions, including current and proposed channel depth, seasonality changes, and extreme weather events. Then…

Context-Aware Search and Retrieval Over Erasure Channels
Sara Ghasvarianjahromi, Yauhen Yakimenka, J\"org Kliewer
https://arxiv.org/abs/2507.11894 htt…

Context-Aware Search and Retrieval Over Erasure Channels
This paper introduces and analyzes a search and retrieval model that adopts key semantic communication principles from retrieval-augmented generation. We specifically present an information-theoretic analysis of a remote document retrieval system operating over a symbol erasure channel. The proposed model encodes the feature vector of a query, derived from term-frequency weights of a language corpus by using a repetition code with an adaptive rate dependent on the contextual importance of the t…

Treasure Hunt: Real-time Targeting of the Long Tail using Training-Time Markers
Daniel D'souza, Julia Kreutzer, Adrien Morisot, Ahmet \"Ust\"un, Sara Hooker
https://arxiv.org/abs/2506.14702

Treasure Hunt: Real-time Targeting of the Long Tail using Training-Time Markers
One of the most profound challenges of modern machine learning is performing well on the long-tail of rare and underrepresented features. Large general-purpose models are trained for many tasks, but work best on high-frequency use cases. After training, it is hard to adapt a model to perform well on specific use cases underrepresented in the training corpus. Relying on prompt engineering or few-shot examples to maximize the output quality on a particular test case can be frustrating, as models …

Quantifying and Reducing Speaker Heterogeneity within the Common Voice Corpus for Phonetic Analysis
Miao Zhang, Aref Farhadipour, Annie Baker, Jiachen Ma, Bogdan Pricop, Eleanor Chodroff
https://arxiv.org/abs/2506.00733

Quantifying and Reducing Speaker Heterogeneity within the Common Voice Corpus for Phonetic Analysis
With its crosslinguistic and cross-speaker diversity, the Mozilla Common Voice Corpus (CV) has been a valuable resource for multilingual speech technology and holds tremendous potential for research in crosslinguistic phonetics and speech sciences. Properly accounting for speaker variation is, however, key to the theoretical and statistical bases of speech research. While CV provides a client ID as an approximation to a speaker ID, multiple speakers can contribute under the same ID. This study …

Active Learning for Text-to-Speech Synthesis with Informative Sample Collection
Kentaro Seki, Shinnosuke Takamichi, Takaaki Saeki, Hiroshi Saruwatari
https://arxiv.org/abs/2507.08319

Active Learning for Text-to-Speech Synthesis with Informative Sample Collection
The construction of high-quality datasets is a cornerstone of modern text-to-speech (TTS) systems. However, the increasing scale of available data poses significant challenges, including storage constraints. To address these issues, we propose a TTS corpus construction method based on active learning. Unlike traditional feed-forward and model-agnostic corpus construction approaches, our method iteratively alternates between data collection and model training, thereby focusing on acquiring data …

What Neuroscience Can Teach AI About Learning in Continuously Changing Environments
Daniel Durstewitz, Bruno Averbeck, Georgia Koppe
https://arxiv.org/abs/2507.02103

What Neuroscience Can Teach AI About Learning in Continuously Changing Environments
Modern AI models, such as large language models, are usually trained once on a huge corpus of data, potentially fine-tuned for a specific task, and then deployed with fixed parameters. Their training is costly, slow, and gradual, requiring billions of repetitions. In stark contrast, animals continuously adapt to the ever-changing contingencies in their environments. This is particularly important for social species, where behavioral policies and reward outcomes may frequently change in interact…

This https://arxiv.org/abs/2506.06266 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Cartridges: Lightweight and general-purpose long context representations via self-study
Large language models are often used to answer queries grounded in large text corpora (e.g. codebases, legal documents, or chat histories) by placing the entire corpus in the context window and leveraging in-context learning (ICL). Although current models support contexts of 100K-1M tokens, this setup is costly to serve because the memory consumption of the KV cache scales with input length. We explore an alternative: training a smaller KV cache offline on each corpus. At inference time, we loa…

Revealing the Hidden Temporal Structure of HubertSoft Embeddings based on the Russian Phonetic Corpus
Anastasia Ananeva, Anton Tomilov, Marina Volkova
https://arxiv.org/abs/2507.06794

Revealing the Hidden Temporal Structure of HubertSoft Embeddings based on the Russian Phonetic Corpus
Self-supervised learning (SSL) models such as Wav2Vec 2.0 and HuBERT have shown remarkable success in extracting phonetic information from raw audio without labelled data. While prior work has demonstrated that SSL embeddings encode phonetic features at the frame level, it remains unclear whether these models preserve temporal structure, specifically, whether embeddings at phoneme boundaries reflect the identity and order of adjacent phonemes. This study investigates the extent to which boundar…

StylOch at PAN: Gradient-Boosted Trees with Frequency-Based Stylometric Features
Jeremi K. Ochab, Mateusz Matias, Tymoteusz Boba, Tomasz Walkowiak
https://arxiv.org/abs/2507.12064

StylOch at PAN: Gradient-Boosted Trees with Frequency-Based Stylometric Features
This submission to the binary AI detection task is based on a modular stylometric pipeline, where: public spaCy models are used for text preprocessing (including tokenisation, named entity recognition, dependency parsing, part-of-speech tagging, and morphology annotation) and extracting several thousand features (frequencies of n-grams of the above linguistic annotations); light-gradient boosting machines are used as the classifier. We collect a large corpus of more than 500 000 machine-generat…

Joint-GCG: Unified Gradient-Based Poisoning Attacks on Retrieval-Augmented Generation Systems
Haowei Wang, Rupeng Zhang, Junjie Wang, Mingyang Li, Yuekai Huang, Dandan Wang, Qing Wang
https://arxiv.org/abs/2506.06151

Joint-GCG: Unified Gradient-Based Poisoning Attacks on Retrieval-Augmented Generation Systems
Retrieval-Augmented Generation (RAG) systems enhance Large Language Models (LLMs) by retrieving relevant documents from external corpora before generating responses. This approach significantly expands LLM capabilities by leveraging vast, up-to-date external knowledge. However, this reliance on external knowledge makes RAG systems vulnerable to corpus poisoning attacks that manipulate generated outputs via poisoned document injection. Existing poisoning attack strategies typically treat the ret…

EraRAG: Efficient and Incremental Retrieval Augmented Generation for Growing Corpora
Fangyuan Zhang, Zhengjun Huang, Yingli Zhou, Qintian Guo, Zhixun Li, Wensheng Luo, Di Jiang, Yixiang Fang, Xiaofang Zhou
https://arxiv.org/abs/2506.20963

EraRAG: Efficient and Incremental Retrieval Augmented Generation for Growing Corpora
Graph-based Retrieval-Augmented Generation (Graph-RAG) enhances large language models (LLMs) by structuring retrieval over an external corpus. However, existing approaches typically assume a static corpus, requiring expensive full-graph reconstruction whenever new documents arrive, limiting their scalability in dynamic, evolving environments. To address these limitations, we introduce EraRAG, a novel multi-layered Graph-RAG framework that supports efficient and scalable dynamic updates. Our met…

SLEEPING-DISCO 9M: A large-scale pre-training dataset for generative music modeling
Tawsif Ahmed, Andrej Radonjic, Gollam Rabby
https://arxiv.org/abs/2506.14293

SLEEPING-DISCO 9M: A large-scale pre-training dataset for generative music modeling
We present Sleeping-DISCO 9M, a large-scale pre-training dataset for music and song. To the best of our knowledge, there are no open-source high-quality dataset representing popular and well-known songs for generative music modeling tasks such as text-music, music-captioning, singing-voice synthesis, melody reconstruction and cross-model retrieval. Past contributions focused on isolated and constrained factors whose core perspective was to create synthetic or re-recorded music corpus (e.g. GTSi…

DocPolarBERT: A Pre-trained Model for Document Understanding with Relative Polar Coordinate Encoding of Layout Structures
Benno Uthayasooriyar, Antoine Ly, Franck Vermet, Caio Corro
https://arxiv.org/abs/2507.08606

DocPolarBERT: A Pre-trained Model for Document Understanding with Relative Polar Coordinate Encoding of Layout Structures
We introduce DocPolarBERT, a layout-aware BERT model for document understanding that eliminates the need for absolute 2D positional embeddings. We extend self-attention to take into account text block positions in relative polar coordinate system rather than the Cartesian one. Despite being pre-trained on a dataset more than six times smaller than the widely used IIT-CDIP corpus, DocPolarBERT achieves state-of-the-art results. These results demonstrate that a carefully designed attention mechan…

Reinforcement Fine-Tuning for Reasoning towards Multi-Step Multi-Source Search in Large Language Models
Wentao Shi, Yiqing Shen
https://arxiv.org/abs/2506.08352

Reinforcement Fine-Tuning for Reasoning towards Multi-Step Multi-Source Search in Large Language Models
Large language models (LLMs) can face factual limitations when responding to time-sensitive queries about recent events that arise after their knowledge thresholds in the training corpus. Existing search-augmented approaches fall into two categories, each with distinct limitations: multi-agent search frameworks incur substantial computational overhead by separating search planning and response synthesis across multiple LLMs, while single-LLM tool-calling methods restrict themselves to sequentia…

Enhancing Essay Cohesion Assessment: A Novel Item Response Theory Approach
Bruno Alexandre Rosa, Hil\'ario Oliveira, Luiz Rodrigues, Eduardo Araujo Oliveira, Rafael Ferreira Mello
https://arxiv.org/abs/2507.08487

Enhancing Essay Cohesion Assessment: A Novel Item Response Theory Approach
Essays are considered a valuable mechanism for evaluating learning outcomes in writing. Textual cohesion is an essential characteristic of a text, as it facilitates the establishment of meaning between its parts. Automatically scoring cohesion in essays presents a challenge in the field of educational artificial intelligence. The machine learning algorithms used to evaluate texts generally do not consider the individual characteristics of the instances that comprise the analysed corpus. In this…

This https://arxiv.org/abs/2506.01495 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

CVC: A Large-Scale Chinese Value Rule Corpus for Value Alignment of Large Language Models
Ensuring that Large Language Models (LLMs) align with mainstream human values and ethical norms is crucial for the safe and sustainable development of AI. Current value evaluation and alignment are constrained by Western cultural bias and incomplete domestic frameworks reliant on non-native rules; furthermore, the lack of scalable, rule-driven scenario generation methods makes evaluations costly and inadequate across diverse cultural contexts. To address these challenges, we propose a hierarchi…

This https://arxiv.org/abs/2503.06474 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIR_…

ROGRAG: A Robustly Optimized GraphRAG Framework
Large language models (LLMs) commonly struggle with specialized or emerging topics which are rarely seen in the training corpus. Graph-based retrieval-augmented generation (GraphRAG) addresses this by structuring domain knowledge as a graph for dynamic retrieval. However, existing pipelines involve complex engineering workflows, making it difficult to isolate the impact of individual components. It is also challenging to evaluate the retrieval effectiveness due to the overlap between the pretra…

SimClass: A Classroom Speech Dataset Generated via Game Engine Simulation For Automatic Speech Recognition Research
Ahmed Adel Attia, Jing Liu, Carl Espy-Wilson
https://arxiv.org/abs/2506.09206

SimClass: A Classroom Speech Dataset Generated via Game Engine Simulation For Automatic Speech Recognition Research
The scarcity of large-scale classroom speech data has hindered the development of AI-driven speech models for education. Public classroom datasets remain limited, and the lack of a dedicated classroom noise corpus prevents the use of standard data augmentation techniques. In this paper, we introduce a scalable methodology for synthesizing classroom noise using game engines, a framework that extends to other domains. Using this methodology, we present SimClass, a dataset that includes both a s…

Narrative Shift Detection: A Hybrid Approach of Dynamic Topic Models and Large Language Models
Kai-Robin Lange, Tobias Schmidt, Matthias Reccius, Henrik M\"uller, Michael Roos, Carsten Jentsch
https://arxiv.org/abs/2506.20269

Narrative Shift Detection: A Hybrid Approach of Dynamic Topic Models and Large Language Models
With rapidly evolving media narratives, it has become increasingly critical to not just extract narratives from a given corpus but rather investigate, how they develop over time. While popular narrative extraction methods such as Large Language Models do well in capturing typical narrative elements or even the complex structure of a narrative, applying them to an entire corpus comes with obstacles, such as a high financial or computational cost. We propose a combination of the language understa…

Hello Afrika: Speech Commands in Kinyarwanda
George Igwegbe, Martins Awojide, Mboh Bless, Nirel Kadzo
https://arxiv.org/abs/2507.01024 https://

Hello Afrika: Speech Commands in Kinyarwanda
Voice or Speech Commands are a subset of the broader Spoken Word Corpus of a language which are essential for non-contact control of and activation of larger AI systems in devices used in everyday life especially for persons with disabilities. Currently, there is a dearth of speech command models for African languages. The Hello Afrika project aims to address this issue and its first iteration is focused on the Kinyarwanda language since the country has shown interest in developing speech recog…

Text to model via SysML: Automated generation of dynamical system computational models from unstructured natural language text via enhanced System Modeling Language diagrams
Matthew Anderson Hendricks, Alice Cicirello
https://arxiv.org/abs/2507.06803

Text to model via SysML: Automated generation of dynamical system computational models from unstructured natural language text via enhanced System Modeling Language diagrams
This paper contributes to speeding up the design and deployment of engineering dynamical systems by proposing a strategy for exploiting domain and expert knowledge for the automated generation of dynamical system computational model starting from a corpus of document relevant to the dynamical system of interest and an input document describing the specific system. This strategy is implemented in five steps and, crucially, it uses system modeling language diagrams (SysML) to extract accurate inf…

This https://arxiv.org/abs/2505.12574 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIR_…

PoisonArena: Uncovering Competing Poisoning Attacks in Retrieval-Augmented Generation
Retrieval-Augmented Generation (RAG) systems, widely used to improve the factual grounding of large language models (LLMs), are increasingly vulnerable to poisoning attacks, where adversaries inject manipulated content into the retriever's corpus. While prior research has predominantly focused on single-attacker settings, real-world scenarios often involve multiple, competing attackers with conflicting objectives. In this work, we introduce PoisonArena, the first benchmark to systematically stu…

SIGIR 2025 -- LiveRAG Challenge Report
David Carmel, Simone Filice, Guy Horowitz, Yoelle Maarek, Oren Somekh, Ran Tavory
https://arxiv.org/abs/2507.04942 h…

SIGIR 2025 -- LiveRAG Challenge Report
The LiveRAG Challenge at SIGIR 2025, held between March and May 2025, provided a competitive platform for advancing Retrieval-Augmented Generation (RAG) technologies. Participants from academia and industry were invited to develop a RAG-based question-answering system using a fixed corpus (Fineweb-10BT) and a common open-source LLM (Falcon3-10B-Instruct). The goal was to facilitate challenging comparisons of retrieval and prompting strategies. During the Live Challenge Day, 70 teams from 27 dif…

When LLMs Disagree: Diagnosing Relevance Filtering Bias and Retrieval Divergence in SDG Search
William A. Ingram, Bipasha Banerjee, Edward A. Fox
https://arxiv.org/abs/2507.02139 …

When LLMs Disagree: Diagnosing Relevance Filtering Bias and Retrieval Divergence in SDG Search
Large language models (LLMs) are increasingly used to assign document relevance labels in information retrieval pipelines, especially in domains lacking human-labeled data. However, different models often disagree on borderline cases, raising concerns about how such disagreement affects downstream retrieval. This study examines labeling disagreement between two open-weight LLMs, LLaMA and Qwen, on a corpus of scholarly abstracts related to Sustainable Development Goals (SDGs) 1, 3, and 7. We is…

Query Drift Compensation: Enabling Compatibility in Continual Learning of Retrieval Embedding Models
Dipam Goswami, Liying Wang, Bart{\l}omiej Twardowski, Joost van de Weijer
https://arxiv.org/abs/2506.00037

Query Drift Compensation: Enabling Compatibility in Continual Learning of Retrieval Embedding Models
Text embedding models enable semantic search, powering several NLP applications like Retrieval Augmented Generation by efficient information retrieval (IR). However, text embedding models are commonly studied in scenarios where the training data is static, thus limiting its applications to dynamic scenarios where new training data emerges over time. IR methods generally encode a huge corpus of documents to low-dimensional embeddings and store them in a database index. During retrieval, a semant…

Data interference: emojis, homoglyphs, and issues of data fidelity in corpora and their results
Matteo Di Cristofaro
https://arxiv.org/abs/2507.01764 https…

Data interference: emojis, homoglyphs, and issues of data fidelity in corpora and their results
Tokenisation - "the process of splitting text into atomic parts" (Brezina & Timperley, 2017: 1) - is a crucial step for corpus linguistics, as it provides the basis for any applicable quantitative method (e.g. collocations) while ensuring the reliability of qualitative approaches. This paper examines how discrepancies in tokenisation affect the representation of language data and the validity of analytical findings: investigating the challenges posed by emojis and homoglyphs, the study highligh…

UiS-IAI@LiveRAG: Retrieval-Augmented Information Nugget-Based Generation of Responses
Weronika {\L}ajewska, Ivica Kostric, Gabriel Iturra-Bocaz, Mariam Arustashvili, Krisztian Balog
https://arxiv.org/abs/2506.22210

UiS-IAI@LiveRAG: Retrieval-Augmented Information Nugget-Based Generation of Responses
Retrieval-augmented generation (RAG) faces challenges related to factual correctness, source attribution, and response completeness. The LiveRAG Challenge hosted at SIGIR'25 aims to advance RAG research using a fixed corpus and a shared, open-source LLM. We propose a modular pipeline that operates on information nuggets-minimal, atomic units of relevant information extracted from retrieved documents. This multistage pipeline encompasses query rewriting, passage retrieval and reranking, nugget d…

This https://arxiv.org/abs/2505.22299 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIR_…

Logical Consistency is Vital: Neural-Symbolic Information Retrieval for Negative-Constraint Queries
Information retrieval plays a crucial role in resource localization. Current dense retrievers retrieve the relevant documents within a corpus via embedding similarities, which compute similarities between dense vectors mainly depending on word co-occurrence between queries and documents, but overlook the real query intents. Thus, they often retrieve numerous irrelevant documents. Particularly in the scenarios of complex queries such as \emph{negative-constraint queries}, their retrieval perfo…

Knowledge-Aware Diverse Reranking for Cross-Source Question Answering
Tong Zhou
https://arxiv.org/abs/2506.20476 https://arxiv.org/pd…

Knowledge-Aware Diverse Reranking for Cross-Source Question Answering
This paper presents Team Marikarp's solution for the SIGIR 2025 LiveRAG competition. The competition's evaluation set, automatically generated by DataMorgana from internet corpora, encompassed a wide range of target topics, question types, question formulations, audience types, and knowledge organization methods. It offered a fair evaluation of retrieving question-relevant supporting documents from a 15M documents subset of the FineWeb corpus. Our proposed knowledge-aware diverse reranking RAG …