Tootfinder

AI5GTest: AI-Driven Specification-Aware Automated Testing and Validation of 5G O-RAN Components
Abiodun Ganiyu, Pranshav Gajjar, Vijay K Shah
https://arxiv.org/abs/2506.10111

AI5GTest: AI-Driven Specification-Aware Automated Testing and Validation of 5G O-RAN Components
The advent of Open Radio Access Networks (O-RAN) has transformed the telecommunications industry by promoting interoperability, vendor diversity, and rapid innovation. However, its disaggregated architecture introduces complex testing challenges, particularly in validating multi-vendor components against O-RAN ALLIANCE and 3GPP specifications. Existing frameworks, such as those provided by Open Testing and Integration Centres (OTICs), rely heavily on manual processes, are fragmented and prone t…

Towards more efficient quantitative safety validation of residual risk for assisted and automated driving
Daniel Betschinske, Malte Schrimpf, Steven Peters, Kamil Klonecki, Jan Peter Karch, Moritz Lippert
https://arxiv.org/abs/2506.10363

Towards more efficient quantitative safety validation of residual risk for assisted and automated driving
The safety validation of Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS) increasingly demands efficient and reliable methods to quantify residual risk while adhering to international standards such as ISO 21448. Traditionally, Field Operational Testing (FOT) has been pivotal for macroscopic safety validation of automotive driving functions up to SAE automation level 2. However, state-of-the-art derivations for empirical safety demonstrations using FOT often result …

Apropos of yet another conversation today, I’m a big fan of using automation in WCAG testing.
But I also know WCAG well enough to understand the limitations (and lies) the tools.
https://adrianroselli.com/2025/04/automated-wcag-testing-is-grrreat.htm…

Automated WCAG Testing Is Grrreat!
I’m a big fan of using automation in WCAG testing. I use bookmarklets, dev tools, browser features & reporting, and a pile of third-party products from assorted vendors. These save me time and effort, letting me focus on more tricky cases. But… Unfortunately, the marketing machines of some vendors seem…

MBTModelGenerator: A software tool for reverse engineering of Model-based Testing (MBT) models from clickstream data of web applications
Sasidhar Matta, Vahid Garousi
https://arxiv.org/abs/2506.08179

MBTModelGenerator: A software tool for reverse engineering of Model-based Testing (MBT) models from clickstream data of web applications
Automated testing has become a standard practice in software engineering, yet the creation of test models and suites remains labor-intensive. To reduce this effort, we developed an open-source tool that automatically generates Model-Based Testing (MBT) models from clickstream data collected during user interaction with web applications. The tool captures UI events, transforms them into state-transition models, and exports the result in a format compatible with the GraphWalker MBT tool. This ena…

LLMs Cannot Reliably Judge (Yet?): A Comprehensive Assessment on the Robustness of LLM-as-a-Judge
Songze Li, Chuokun Xu, Jiaying Wang, Xueluan Gong, Chen Chen, Jirui Zhang, Jun Wang, Kwok-Yan Lam, Shouling Ji
https://arxiv.org/abs/2506.09443

LLMs Cannot Reliably Judge (Yet?): A Comprehensive Assessment on the Robustness of LLM-as-a-Judge
Large Language Models (LLMs) have demonstrated remarkable intelligence across various tasks, which has inspired the development and widespread adoption of LLM-as-a-Judge systems for automated model testing, such as red teaming and benchmarking. However, these systems are susceptible to adversarial attacks that can manipulate evaluation outcomes, raising concerns about their robustness and, consequently, their trustworthiness. Existing evaluation methods adopted by LLM-based judges are often pie…

LLM-Guided Scenario-based GUI Testing
Shengcheng Yu, Yuchen Ling, Chunrong Fang, Quan Zhou, Chunyang Chen, Shaomin Zhu, Zhenyu Chen
https://arxiv.org/abs/2506.05079

LLM-Guided Scenario-based GUI Testing
The assurance of mobile app GUI is more and more significant. Automated GUI testing approaches of different strategies have been developed, while there are still huge gaps between the approaches and the app business logic, not taking the completion of specific testing scenarios as the exploration target, leading to the exploration missing of critical app functionalities. Learning from the manual testing, which takes testing scenarios with app business logic as the basic granularity, in this pap…

Series C, Episode 04 - Dawn of the Gods
AVON: Orac has a weakness: a thirst for knowledge. Infallibility depends upon your point of view.
TARRANT: How about testing his infallibility in the recycling machine?
https://blake.torpidity.net/m/304/31 B7B4

Blakes7Bot | Search
Companion site to the Blakes7Bot Twitter, Mastodon, and Bluesky accounts. Automated lines of Blakes 7 dialog. Includes a script search facility.

Temac: Multi-Agent Collaboration for Automated Web GUI Testing
Chenxu Liu, Zhiyu Gu, Guoquan Wu, Ying Zhang, Jun Wei, Tao Xie
https://arxiv.org/abs/2506.00520

Temac: Multi-Agent Collaboration for Automated Web GUI Testing
Quality assurance of web applications is critical, as web applications play an essential role in people's daily lives. To reduce labor costs, automated web GUI testing (AWGT) is widely adopted, exploring web applications via GUI actions such as clicks and text inputs. However, these approaches face limitations in generating continuous and meaningful action sequences capable of covering complex functionalities. Recent work incorporates large language models (LLMs) for GUI testing. However, these…

This https://arxiv.org/abs/2411.05185 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCR_…

PentestAgent: Incorporating LLM Agents to Automated Penetration Testing
Penetration testing is a critical technique for identifying security vulnerabilities, traditionally performed manually by skilled security specialists. This complex process involves gathering information about the target system, identifying entry points, exploiting the system, and reporting findings. Despite its effectiveness, manual penetration testing is time-consuming and expensive, often requiring significant expertise and resources that many organizations cannot afford. While automated pen…

Automated Web Application Testing: End-to-End Test Case Generation with Large Language Models and Screen Transition Graphs
Nguyen-Khang Le, Quan Minh Bui, Minh Ngoc Nguyen, Hiep Nguyen, Trung Vo, Son T. Luu, Shoshin Nomura, Minh Le Nguyen
https://arxiv.org/abs/2506.02529

Automated Web Application Testing: End-to-End Test Case Generation with Large Language Models and Screen Transition Graphs
Web applications are critical to modern software ecosystems, yet ensuring their reliability remains challenging due to the complexity and dynamic nature of web interfaces. Recent advances in large language models (LLMs) have shown promise in automating complex tasks, but limitations persist in handling dynamic navigation flows and complex form interactions. This paper presents an automated system for generating test cases for two key aspects of web application testing: site navigation and form …

VISCA: Inferring Component Abstractions for Automated End-to-End Testing
Parsa Alian, Martin Tang, Ali Mesbah
https://arxiv.org/abs/2506.04161 https://

VISCA: Inferring Component Abstractions for Automated End-to-End Testing
Providing optimal contextual input presents a significant challenge for automated end-to-end (E2E) test generation using large language models (LLMs), a limitation that current approaches inadequately address. This paper introduces Visual-Semantic Component Abstractor (VISCA), a novel method that transforms webpages into a hierarchical, semantically rich component abstraction. VISCA starts by partitioning webpages into candidate segments utilizing a novel heuristic-based segmentation method. Th…

This https://arxiv.org/abs/2506.02943 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSE_…

A Multi-agent LLM-based JUit Test Generation with Strong Oracles
Unit testing plays a critical role in ensuring software correctness. However, writing unit tests manually is laborious, especially for strong typed languages like Java, motivating the need for automated approaches. Traditional methods primarily rely on search-based or randomized algorithms to generate tests that achieve high code coverage and produce regression oracles, which are derived from the program's current behavior rather than its intended functionality. Recent advances in large languag…

This https://arxiv.org/abs/2408.09568 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSE_…

MergeRepair: An Exploratory Study on Merging Task-Specific Adapters in Code LLMs for Automated Program Repair
Large Language Models (LLMs) have shown high capabilities in several software development-related tasks such as program repair, documentation, code refactoring, debugging, and testing. However, training these models requires massive amount of data and significant computational resources. Adapters are specialized, small modules designed for parameter efficient fine-tuning of LLMs for specific tasks, domains, or applications without requiring extensive retraining of the entire model. These adapte…

This https://arxiv.org/abs/2506.02943 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSE_…

A Multi-agent LLM-based JUit Test Generation with Strong Oracles
Unit testing plays a critical role in ensuring software correctness. However, writing unit tests manually is laborious, especially for strong typed languages like Java, motivating the need for automated approaches. Traditional methods primarily rely on search-based or randomized algorithms to generate tests that achieve high code coverage and produce regression oracles, which are derived from the program's current behavior rather than its intended functionality. Recent advances in large languag…

A Multi-agent LLM-based JUit Test Generation with Strong Oracles
Qinghua Xu, Guancheng Wang, Lionel Briand, Kui Liu
https://arxiv.org/abs/2506.02943 https:…

A Multi-agent LLM-based JUit Test Generation with Strong Oracles
Unit testing plays a critical role in ensuring software correctness. However, writing unit tests manually is laborious, especially for strong typed languages like Java, motivating the need for automated approaches. Traditional methods primarily rely on search-based or randomized algorithms to generate tests that achieve high code coverage and produce regression oracles, which are derived from the program's current behavior rather than its intended functionality. Recent advances in large languag…