Tootfinder

UK AI Security Institute report: AI models are rapidly improving at potentially dangerous biological and chemical tasks, and show fast jumps in self-replication (Shakeel Hashim/Transformer)
https://www.transformernews.ai/p/aisi-ai-s

AI is making dangerous lab work accessible to novices, UK’s AISI finds
UK AISI’s first Frontier AI Trends Report finds that AI models are getting better at self-replication, too

Credal Transformer: A Principled Approach for Quantifying and Mitigating Hallucinations in Large Language Models
Shihao Ji, Zihui Song, Jiajie Huang
https://arxiv.org/abs/2510.12137

Credal Transformer: A Principled Approach for Quantifying and Mitigating Hallucinations in Large Language Models
Large Language Models (LLMs) hallucinate, generating factually incorrect yet confident assertions. We argue this stems from the Transformer's Softmax function, which creates "Artificial Certainty" by collapsing ambiguous attention scores into a single probability distribution, discarding uncertainty information at each layer. To fix this, we introduce the Credal Transformer, which replaces standard attention with a Credal Attention Mechanism (CAM) based on evidential theory. CAM produces a "cre…

Hybrid Explanation-Guided Learning for Transformer-Based Chest X-Ray Diagnosis
Shelley Zixin Shu, Haozhe Luo, Alexander Poellinger, Mauricio Reyes
https://arxiv.org/abs/2510.12704

Hybrid Explanation-Guided Learning for Transformer-Based Chest X-Ray Diagnosis
Transformer-based deep learning models have demonstrated exceptional performance in medical imaging by leveraging attention mechanisms for feature representation and interpretability. However, these models are prone to learning spurious correlations, leading to biases and limited generalization. While human-AI attention alignment can mitigate these issues, it often depends on costly manual supervision. In this work, we propose a Hybrid Explanation-Guided Learning (H-EGL) framework that combines…

Hybrid Vision Transformer and Quantum Convolutional Neural Network for Image Classification
Mingzhu Wang, Yun Shang
https://arxiv.org/abs/2510.12291 https://

Hybrid Vision Transformer and Quantum Convolutional Neural Network for Image Classification
Quantum machine learning (QML) holds promise for computational advantage, yet progress on real-world tasks is hindered by classical preprocessing and noisy devices. We introduce ViT-QCNN-FT, a hybrid framework that integrates a fine-tuned Vision Transformer with a quantum convolutional neural network (QCNN) to compress high-dimensional images into features suited for noisy intermediate-scale quantum (NISQ) devices. By systematically probing entanglement, we show that ansatzes with uniformly dis…

Audio Palette: A Diffusion Transformer with Multi-Signal Conditioning for Controllable Foley Synthesis
Junnuo Wang
https://arxiv.org/abs/2510.12175 https://

Audio Palette: A Diffusion Transformer with Multi-Signal Conditioning for Controllable Foley Synthesis
Recent advances in diffusion-based generative models have enabled high-quality text-to-audio synthesis, but fine-grained acoustic control remains a significant challenge in open-source research. We present Audio Palette, a diffusion transformer (DiT) based model that extends the Stable Audio Open architecture to address this "control gap" in controllable audio generation. Unlike prior approaches that rely solely on semantic conditioning, Audio Palette introduces four time-varying control signal…

Software Defect Prediction using Autoencoder Transformer Model
Seshu Barma, Mohanakrishnan Hariharan, Satish Arvapalli
https://arxiv.org/abs/2510.10840 https://

Software Defect Prediction using Autoencoder Transformer Model
An AI-ML-powered quality engineering approach uses AI-ML to enhance software quality assessments by predicting defects. Existing ML models struggle with noisy data types, imbalances, pattern recognition, feature extraction, and generalization. To address these challenges, we develop a new model, Adaptive Differential Evolution (ADE) based Quantum Variational Autoencoder-Transformer (QVAET) Model (ADE-QVAET). ADE combines with QVAET to obtain high-dimensional latent features and maintain sequent…

GrifFinNet: A Graph-Relation Integrated Transformer for Financial Predictions
Chenlanhui Dai, Wenyan Wang, Yusi Fan, Yueying Wang, Lan Huang, Kewei Li, Fengfeng Zhou
https://arxiv.org/abs/2510.10387

GrifFinNet: A Graph-Relation Integrated Transformer for Financial Predictions
Predicting stock returns remains a central challenge in quantitative finance, transitioning from traditional statistical methods to contemporary deep learning techniques. However, many current models struggle with effectively capturing spatio-temporal dynamics and integrating multiple relational data sources. This study proposes GrifFinNet, a Graph-Relation Integrated Transformer for Financial Predictions, which combines multi-relational graph modeling with Transformer-based temporal encoding. …

Nvidia launches Nemotron 3, a family of AI models using a hybrid mixture-of-experts architecture and the Mamba-Transformer design, in 30B, 100B, and ~500B sizes (Emilia David/VentureBeat)
https://venturebeat.com/ai/nvidia-debuts-nemotron-3-w…

Efficient Autoregressive Inference for Transformer Probabilistic Models
Conor Hassan, Nasrulloh Loka, Cen-You Li, Daolang Huang, Paul E. Chang, Yang Yang, Francesco Silvestrin, Samuel Kaski, Luigi Acerbi
https://arxiv.org/abs/2510.09477

Efficient Autoregressive Inference for Transformer Probabilistic Models
Transformer-based models for amortized probabilistic inference, such as neural processes, prior-fitted networks, and tabular foundation models, excel at single-pass marginal prediction. However, many real-world applications, from signal interpolation to multi-column tabular predictions, require coherent joint distributions that capture dependencies between predictions. While purely autoregressive architectures efficiently generate such distributions, they sacrifice the flexible set-conditioning…

From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics
Zheng-An Chen, Tao Luo
https://arxiv.org/abs/2510.06954 https://a…

From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics
Although transformer-based models have shown exceptional empirical performance, the fundamental principles governing their training dynamics are inadequately characterized beyond configuration-specific studies. Inspired by empirical evidence showing improved reasoning capabilities under small initialization scales in language models, we employ the gradient flow analytical framework established in [Zhou et al. NeurIPS 2022] to systematically investigate linearized Transformer training dynamics. …

"Für jede Trafostation eine LEG" [‘A local energy community for every transformer station’]
It is fascinating how local energy communities are encouraged in Switzerland (so not governed by EU law, that also creates that role), and how the (local) utilities are also actively involved in this.
https://www.

"Für jede Trafostation eine LEG"
Winterthur - Stadtwerk Winterthur will mit lokalen Elektrizitätsgemeinschaften vor allem zur Erreichung der städtischen Klima- und Energieziele beitragen. Produktmanager Roland Hongler erklärt, wie das Modell gestaltet ist: Für die Teilnehmenden sollte es attraktiv sein und für das Stadtwerk zugleich wirtschaftlich tragfähig bleiben.

Crosslisted article(s) found for cs.AI. https://arxiv.org/list/cs.AI/new
[6/6]:
- Hybrid Explanation-Guided Learning for Transformer-Based Chest X-Ray Diagnosis
Shelley Zixin Shu, Haozhe Luo, Alexander Poellinger, Mauricio Reyes

ILD-VIT: A Unified Vision Transformer Architecture for Detection of Interstitial Lung Disease from Respiratory Sounds
Soubhagya Ranjan Hota, Arka Roy, Udit Satija
https://arxiv.org/abs/2510.11458

ILD-VIT: A Unified Vision Transformer Architecture for Detection of Interstitial Lung Disease from Respiratory Sounds
Interstitial lung disease (ILD) represents a group of restrictive chronic pulmonary diseases that impair oxygen acquisition by causing irreversible changes in the lungs such as fibrosis, scarring of parenchyma, etc. ILD conditions are often diagnosed by various clinical modalities such as spirometry, high-resolution lung imaging techniques, crackling respiratory sounds (RSs), etc. In this letter, we develop a novel vision transformer (VIT)-based deep learning framework namely, ILD-VIT, to detec…

VER: Vision Expert Transformer for Robot Learning via Foundation Distillation and Dynamic Routing
Yixiao Wang, Mingxiao Huo, Zhixuan Liang, Yushi Du, Lingfeng Sun, Haotian Lin, Jinghuan Shang, Chensheng Peng, Mohit Bansal, Mingyu Ding, Masayoshi Tomizuka
https://arxiv.org/abs/2510.05213

VER: Vision Expert Transformer for Robot Learning via Foundation Distillation and Dynamic Routing
Pretrained vision foundation models (VFMs) advance robotic learning via rich visual representations, yet individual VFMs typically excel only in specific domains, limiting generality across tasks. Distilling multiple VFMs into a unified representation for policy can mitigate this limitation but often yields inflexible task-specific feature selection and requires costly full re-training to incorporate robot-domain knowledge. We propose VER, a Vision Expert transformer for Robot learning. During …

An Encoder-Integrated PhoBERT with Graph Attention for Vietnamese Token-Level Classification
Ba-Quang Nguyen
https://arxiv.org/abs/2510.11537 https://arxiv…

An Encoder-Integrated PhoBERT with Graph Attention for Vietnamese Token-Level Classification
We propose a novel neural architecture named TextGraphFuseGAT, which integrates a pretrained transformer encoder (PhoBERT) with Graph Attention Networks for token-level classification tasks. The proposed model constructs a fully connected graph over the token embeddings produced by PhoBERT, enabling the GAT layer to capture rich inter-token dependencies beyond those modeled by sequential context alone. To further enhance contextualization, a Transformer-style self-attention layer is applied on …

Latent Representation Learning in Heavy-Ion Collisions with MaskPoint Transformer
Jing-Zong Zhang, Shuang Guo, Li-Lin Zhu, Lingxiao Wang, Guo-Liang Ma
https://arxiv.org/abs/2510.06691

Latent Representation Learning in Heavy-Ion Collisions with MaskPoint Transformer
A central challenge in high-energy nuclear physics is to extract informative features from the high-dimensional final-state data of heavy-ion collisions (HIC) in order to enable reliable downstream analyses. Traditional approaches often rely on selected observables, which may miss subtle but physically relevant structures in the data. To address this, we introduce a Transformer-based autoencoder trained with a two-stage paradigm: self-supervised pre-training followed by supervised fine-tuning. …

A Scalable AI Driven, IoT Integrated Cognitive Digital Twin for Multi-Modal Neuro-Oncological Prognostics and Tumor Kinetics Prediction using Enhanced Vision Transformer and XAI
Saptarshi Banerjee, Himadri Nath Saha, Utsho Banerjee, Rajarshi Karmakar, Jon Turdiev
https://arxiv.org/abs/2510.05123

A Scalable AI Driven, IoT Integrated Cognitive Digital Twin for Multi-Modal Neuro-Oncological Prognostics and Tumor Kinetics Prediction using Enhanced Vision Transformer and XAI
Neuro-oncological prognostics are now vital in modern clinical neuroscience because brain tumors pose significant challenges in detection and management. To tackle this issue, we propose a cognitive digital twin framework that combines real-time EEG signals from a wearable skullcap with structural MRI data for dynamic and personalized tumor monitoring. At the heart of this framework is an Enhanced Vision Transformer (ViT++) that includes innovative components like Patch-Level Attention Regulari…

LadderSym: A Multimodal Interleaved Transformer for Music Practice Error Detection
Benjamin Shiue-Hal Chou, Purvish Jajal, Nick John Eliopoulos, James C. Davis, George K. Thiruvathukal, Kristen Yeon-Ji Yun, Yung-Hsiang Lu
https://arxiv.org/abs/2510.08580

LadderSym: A Multimodal Interleaved Transformer for Music Practice Error Detection
Music learners can greatly benefit from tools that accurately detect errors in their practice. Existing approaches typically compare audio recordings to music scores using heuristics or learnable models. This paper introduces \textit{LadderSym}, a novel Transformer-based method for music error detection. \textit{LadderSym} is guided by two key observations about the state-of-the-art approaches: (1) late fusion limits inter-stream alignment and cross-modality comparison capability; and (2) relia…

Locality-Sensitive Hashing-Based Efficient Point Transformer for Charged Particle Reconstruction
Shitij Govil, Jack P. Rodgers, Yuan-Tang Chou, Siqi Miao, Amit Saha, Advaith Anand, Kilian Lieret, Gage DeZoort, Mia Liu, Javier Duarte, Pan Li, Shih-Chieh Hsu
https://arxiv.org/abs/2510.07594

Locality-Sensitive Hashing-Based Efficient Point Transformer for Charged Particle Reconstruction
Charged particle track reconstruction is a foundational task in collider experiments and the main computational bottleneck in particle reconstruction. Graph neural networks (GNNs) have shown strong performance for this problem, but costly graph construction, irregular computations, and random memory access patterns substantially limit their throughput. The recently proposed Hashing-based Efficient Point Transformer (HEPT) offers a theoretically guaranteed near-linear complexity for large point …

LRQ-Solver: A Transformer-Based Neural Operator for Fast and Accurate Solving of Large-scale 3D PDEs
Peijian Zeng, Guan Wang, Haohao Gu, Xiaoguang Hu, TiezhuGao, Zhuowei Wang, Aimin Yang, Xiaoyu Song
https://arxiv.org/abs/2510.11636

LRQ-Solver: A Transformer-Based Neural Operator for Fast and Accurate Solving of Large-scale 3D PDEs
Solving large-scale Partial Differential Equations (PDEs) on complex three-dimensional geometries represents a central challenge in scientific and engineering computing, often impeded by expensive pre-processing stages and substantial computational overhead. We introduce Low-Rank Query-based PDE Solver (LRQ-Solver), a physics-integrated framework engineered for rapid, accurate, and highly scalable simulations of industrial-grade models. This framework is built upon two primary technical innovat…

Self-attention enabled quantum path analysis of high-harmonic generation in solids
Cong Zhao, Xiaozhou Zou
https://arxiv.org/abs/2510.12443 https://arxiv.o…

Self-attention enabled quantum path analysis of high-harmonic generation in solids
High-harmonic generation (HHG) in solids provides a powerful platform to probe ultrafast electron dynamics and interband--intraband coupling. However, disentangling the complex many-body contributions in the HHG spectrum remains challenging. Here we introduce a machine-learning approach based on a Transformer encoder to analyze and reconstruct HHG signals computed from a one-dimensional Kronig--Penney model. The self-attention mechanism inherently highlights correlations between temporal dipole…

HTMformer: Hybrid Time and Multivariate Transformer for Time Series Forecasting
Tan Wang, Yun Wei Dong, Tao Zhang, Qi Wang
https://arxiv.org/abs/2510.07084 https://

HTMformer: Hybrid Time and Multivariate Transformer for Time Series Forecasting
Transformer-based methods have achieved impressive results in time series forecasting. However, existing Transformers still exhibit limitations in sequence modeling as they tend to overemphasize temporal dependencies. This incurs additional computational overhead without yielding corresponding performance gains. We find that the performance of Transformers is highly dependent on the embedding method used to learn effective representations. To address this issue, we extract multivariate features…

Interplay Between Belief Propagation and Transformer: Differential-Attention Message Passing Transformer
Chin Wa Lau, Xiang Shi, Ziyan Zheng, Haiwen Cao, Nian Guo
https://arxiv.org/abs/2509.15637

Interplay Between Belief Propagation and Transformer: Differential-Attention Message Passing Transformer
Transformer-based neural decoders have emerged as a promising approach to error correction coding, combining data-driven adaptability with efficient modeling of long-range dependencies. This paper presents a novel decoder architecture that integrates classical belief propagation principles with transformer designs. We introduce a differentiable syndrome loss function leveraging global codebook structure and a differential-attention mechanism optimizing bit and syndrome embedding interactions. E…

Sources: the NY governor proposes a rewrite of the RAISE Act, the AI bill that recently passed NY legislature, with text copied verbatim from California's SB 53 (Shakeel Hashim/Transformer)
https://www.transformernews.ai/p/new-york-governor-hochul-raise-act-sb-5…

New York’s governor is trying to turn the RAISE Act into an SB 53 copycat
EXCLUSIVE: Gov. Kathy Hochul is proposing to strike the entire text of the RAISE Act, replacing it with verbatim language from SB 53, sources tell Transformer.

What If : Understanding Motion Through Sparse Interactions
Stefan Andreas Baumann, Nick Stracke, Timy Phan, Bj\"orn Ommer
https://arxiv.org/abs/2510.12777 https://

What If : Understanding Motion Through Sparse Interactions
Understanding the dynamics of a physical scene involves reasoning about the diverse ways it can potentially change, especially as a result of local interactions. We present the Flow Poke Transformer (FPT), a novel framework for directly predicting the distribution of local motion, conditioned on sparse interactions termed "pokes". Unlike traditional methods that typically only enable dense sampling of a single realization of scene dynamics, FPT provides an interpretable directly accessible repr…

ENLighten: Lighten the Transformer, Enable Efficient Optical Acceleration
Hanqing Zhu, Zhican Zhou, Shupeng Ning, Xuhao Wu, Ray Chen, Yating Wan, David Pan
https://arxiv.org/abs/2510.01673

ENLighten: Lighten the Transformer, Enable Efficient Optical Acceleration
Photonic computing has emerged as a promising substrate for accelerating the dense linear-algebra operations at the heart of AI, yet adoption for large Transformer models remains in its infancy. We identify two bottlenecks: (1) costly electro--optic conversions and data-movement overheads that erode energy efficiency as model sizes scale; (2) a mismatch between limited on-chip photonic resources and Transformer scale, which forces frequent reuse of photonic tensor cores and dilutes throughput g…

DiTSinger: Scaling Singing Voice Synthesis with Diffusion Transformer and Implicit Alignment
Zongcai Du, Guilin Deng, Xiaofeng Guo, Xin Gao, Linke Li, Kaichang Cheng, Fubo Han, Siyu Yang, Peng Liu, Pan Zhong, Qiang Fu
https://arxiv.org/abs/2510.09016

DiTSinger: Scaling Singing Voice Synthesis with Diffusion Transformer and Implicit Alignment
Recent progress in diffusion-based Singing Voice Synthesis (SVS) demonstrates strong expressiveness but remains limited by data scarcity and model scalability. We introduce a two-stage pipeline: a compact seed set of human-sung recordings is constructed by pairing fixed melodies with diverse LLM-generated lyrics, and melody-specific models are trained to synthesize over 500 hours of high-quality Chinese singing data. Building on this corpus, we propose DiTSinger, a Diffusion Transformer with Ro…

Crosslisted article(s) found for eess.SP. https://arxiv.org/list/eess.SP/new
[1/1]:
- Soft Graph Transformer for MIMO Detection
Jiadong Hong, Lei Liu, Xinyu Bian, Wenjie Wang, Zhaoyang Zhang

Atlas-free Brain Network Transformer
Shuai Huang, Xuan Kan, James J. Lah, Deqiang Qiu
https://arxiv.org/abs/2510.03306 https://arxiv.org/pdf/2510.03306

Atlas-free Brain Network Transformer
Current atlas-based approaches to brain network analysis rely heavily on standardized anatomical or connectivity-driven brain atlases. However, these fixed atlases often introduce significant limitations, such as spatial misalignment across individuals, functional heterogeneity within predefined regions, and atlas-selection biases, collectively undermining the reliability and interpretability of the derived brain networks. To address these challenges, we propose a novel atlas-free brain network…

Dual-Path Phishing Detection: Integrating Transformer-Based NLP with Structural URL Analysis
Ibrahim Altan, Abdulla Bachir, Yousuf Parbhulkar, Abdul Muksith Rizvi, Moshiur Farazi
https://arxiv.org/abs/2509.20972

Dual-Path Phishing Detection: Integrating Transformer-Based NLP with Structural URL Analysis
Phishing emails pose a persistent and increasingly sophisticated threat, undermining email security through deceptive tactics designed to exploit both semantic and structural vulnerabilities. Traditional detection methods, often based on isolated analysis of email content or embedded URLs, fail to comprehensively address these evolving attacks. In this paper, we propose a dual-path phishing detection framework that integrates transformer-based natural language processing (NLP) with classical ma…

Pretty sure the power supply in this switch is fried. Possibly a short in the coil/transformer next to the hot power regulator.
$8 to replace the entire unit, delivered.

Comparing Symmetrized Determinant Neural Quantum States for the Hubbard Model
Louis Sharma, Ahmedeo Shokry, Rajah Nutakki, Olivier Simard, Michel Ferrero, Filippo Vicentini
https://arxiv.org/abs/2510.11710

Comparing Symmetrized Determinant Neural Quantum States for the Hubbard Model
Accurate simulations of the Hubbard model are crucial to understanding strongly correlated phenomena, where small energy differences between competing orders demand high numerical precision. In this work, Neural Quantum States are used to probe the strongly coupled and underdoped regime of the square-lattice Hubbard model. We systematically compare the Hidden Fermion Determinant State and the Jastrow-Backflow ansatz, parametrized by a Vision Transformer, finding that in practice, their accuracy…

A Comparative Analysis of Contextual Representation Flow in State-Space and Transformer Architectures
Nhat M. Hoang, Do Xuan Long, Cong-Duy Nguyen, Min-Yen Kan, Luu Anh Tuan
https://arxiv.org/abs/2510.06640

A Comparative Analysis of Contextual Representation Flow in State-Space and Transformer Architectures
State Space Models (SSMs) have recently emerged as efficient alternatives to Transformer-Based Models (TBMs) for long-sequence processing, offering linear scaling and lower memory use. Yet, how contextual information flows across layers and tokens in these architectures remains understudied. We present the first unified, token- and layer-level analysis of representation propagation in SSMs and TBMs. Using centered kernel alignment, stability metrics, and probing, we characterize how representat…

From Embeddings to Equations: Genetic-Programming Surrogates for Interpretable Transformer Classification
Mohammad Sadegh Khorshidi, Navid Yazdanjue, Hassan Gharoun, Mohammad Reza Nikoo, Fang Chen, Amir H. Gandomi
https://arxiv.org/abs/2509.21341

From Embeddings to Equations: Genetic-Programming Surrogates for Interpretable Transformer Classification
We study symbolic surrogate modeling of frozen Transformer embeddings to obtain compact, auditable classifiers with calibrated probabilities. For five benchmarks (SST2G, 20NG, MNIST, CIFAR10, MSC17), embeddings from ModernBERT, DINOv2, and SigLIP are partitioned on the training set into disjoint, information-preserving views via semantic-preserving feature partitioning (SPFP). A cooperative multi-population genetic program (MEGP) then learns additive, closed-form logit programs over these views…

Variable Rate Image Compression via N-Gram Context based Swin-transformer
Priyanka Mudgal, Feng Liu
https://arxiv.org/abs/2510.00058 https://arxiv.org/pdf/…

Variable Rate Image Compression via N-Gram Context based Swin-transformer
This paper presents an N-gram context-based Swin Transformer for learned image compression. Our method achieves variable-rate compression with a single model. By incorporating N-gram context into the Swin Transformer, we overcome its limitation of neglecting larger regions during high-resolution image reconstruction due to its restricted receptive field. This enhancement expands the regions considered for pixel restoration, thereby improving the quality of high-resolution reconstructions. Our m…

Transformer Classification of Breast Lesions: The BreastDCEDL_AMBL Benchmark Dataset and 0.92 AUC Baseline
Naomi Fridman (Ariel University), Anat Goldstein (Ariel University)
https://arxiv.org/abs/2509.26440

Transformer Classification of Breast Lesions: The BreastDCEDL_AMBL Benchmark Dataset and 0.92 AUC Baseline
The error is caused by special characters that arXiv's system doesn't recognize. Here's the cleaned version with all problematic characters replaced: Breast magnetic resonance imaging is a critical tool for cancer detection and treatment planning, but its clinical utility is hindered by poor specificity, leading to high false-positive rates and unnecessary biopsies. This study introduces a transformer-based framework for automated classification of breast lesions in dynamic contrast-enhanced MR…

How Pathway, a startup developing an alternative to the transformer, aims to use its Dragon Hatchling architecture to create a new class of adaptive AI systems (Steven Rosenbush/Wall Street Journal)
https://www.wsj.com/articles/an-ai…

PyramidStyler: Transformer-Based Neural Style Transfer with Pyramidal Positional Encoding and Reinforcement Learning
Raahul Krishna Durairaju (California State University, Fullerton), K. Saruladha (Puducherry Technological University)
https://arxiv.org/abs/2510.01715

PyramidStyler: Transformer-Based Neural Style Transfer with Pyramidal Positional Encoding and Reinforcement Learning
Neural Style Transfer (NST) has evolved from Gatys et al.'s (2015) CNN-based algorithm, enabling AI-driven artistic image synthesis. However, existing CNN and transformer-based models struggle to scale efficiently to complex styles and high-resolution inputs. We introduce PyramidStyler, a transformer framework with Pyramidal Positional Encoding (PPE): a hierarchical, multi-scale encoding that captures both local details and global context while reducing computational load. We further incorporat…

This is so much "AI" reporting: Claims about potentials and/or threads. I'd just like to have grown-up conversations about tech again :(
"The actual current user base for evil chatbots is the cyber security vendors, who scaremonger how only their good AI can possibly stop this automated hacker evil!"
(Original title: AI for evil — hacked by WormGPT!)

AI for evil — hacked by WormGPT!
A chatbot is a wrapper around a large language model, an AI transformer model that’s been trained on the whole internet, all the books the AI vendor can find, and all the other text in the world. A…

Signature-Informed Transformer for Asset Allocation
Yoontae Hwang, Stefan Zohren
https://arxiv.org/abs/2510.03129 https://arxiv.org/pdf/2510.03129

Signature-Informed Transformer for Asset Allocation
Robust asset allocation is a key challenge in quantitative finance, where deep-learning forecasters often fail due to objective mismatch and error amplification. We introduce the Signature-Informed Transformer (SIT), a novel framework that learns end-to-end allocation policies by directly optimizing a risk-aware financial objective. SIT's core innovations include path signatures for a rich geometric representation of asset dynamics and a signature-augmented attention mechanism embedding financi…

PAT: Pattern-Perceptive Transformer for Error Detection in Relational Databases
Jian Fu, Xixian Han, Xiaolong Wan, Wenjian Wang
https://arxiv.org/abs/2509.25907 https://

PAT: Pattern-Perceptive Transformer for Error Detection in Relational Databases
Error detection in relational databases is critical for maintaining data quality and is fundamental to tasks such as data cleaning and assessment. Current error detection studies mostly employ the multi-detector approach to handle heterogeneous attributes in databases, incurring high costs. Additionally, their data preprocessing strategies fail to leverage the variable-length characteristic of data sequences, resulting in reduced accuracy. In this paper, we propose an attribute-wise PAttern-per…

CosmoUiT: A Vision Transformer-UNet Hybrid for Fast and Accurate Emulation of 21-cm Maps from the Epoch of Reionization
Prasad Rajesh Posture, Yashrajsinh Mahida, Suman Majumdar, Leon Noble
https://arxiv.org/abs/2510.01121

CosmoUiT: A Vision Transformer-UNet Hybrid for Fast and Accurate Emulation of 21-cm Maps from the Epoch of Reionization
The observation of the redshifted 21-cm signal from the intergalactic medium will probe the epoch of reionization (EoR) with unprecedented detail. Various simulations are being developed and used to predict and understand the nature and morphology of this signal. However, these simulations are computationally very expensive and time-consuming to produce in large numbers. To overcome this problem, an efficient field-level emulator of this signal is required. However, the EoR 21-cm signal is high…

cAItomorph: Transformer-Based Hematological Malignancy Prediction from Peripheral Blood Smears in a Real-Word Cohort
Muhammed Furkan Dasdelen, Ivan Kukuljan, Peter Lienemann, Ario Sadafi, Matthias Hehr, Karsten Spiekermann, Christian Pohlkamp, Carsten Marr
https://arxiv.org/abs/2509.20402

cAItomorph: Transformer-Based Hematological Malignancy Prediction from Peripheral Blood Smears in a Real-Word Cohort
Peripheral blood smears remain a cornerstone in the diagnosis of hematological neoplasms, offering rapid and valuable insights that inform subsequent diagnostic steps. However, since neoplastic transformations typically arise in the bone marrow, they may not manifest as detectable aberrations in peripheral blood, presenting a diagnostic challenge. In this paper, we introduce cAItomorph, an explainable transformer-based AI model, trained to classify hematological malignancies based on peripheral…

#OTD 2013: A time when Tesla still meant progress. One of the first Tesla Superchargers in Europe, in Zevenaar 🇳🇱. Until then, #EV fast chargers were often single devices, but this was one of the first hubs.

Dr.LLM: Dynamic Layer Routing in LLMs
Ahmed Heakl, Martin Gubri, Salman Khan, Sangdoo Yun, Seong Joon Oh
https://arxiv.org/abs/2510.12773 https://arxiv.org…

Dr.LLM: Dynamic Layer Routing in LLMs
Large Language Models (LLMs) process every token through all layers of a transformer stack, causing wasted computation on simple queries and insufficient flexibility for harder ones that need deeper reasoning. Adaptive-depth methods can improve efficiency, but prior approaches rely on costly inference-time search, architectural changes, or large-scale retraining, and in practice often degrade accuracy despite efficiency gains. We introduce Dr.LLM, Dynamic routing of Layers for LLMs, a retrofitt…

Influence-Guided Concolic Testing of Transformer Robustness
Chih-Duo Hong, Yu Wang, Yao-Chen Chang, Fang Yu
https://arxiv.org/abs/2509.23806 https://arxiv.…

Influence-Guided Concolic Testing of Transformer Robustness
Concolic testing for deep neural networks alternates concrete execution with constraint solving to search for inputs that flip decisions. We present an {influence-guided} concolic tester for Transformer classifiers that ranks path predicates by SHAP-based estimates of their impact on the model output. To enable SMT solving on modern architectures, we prototype a solver-compatible, pure-Python semantics for multi-head self-attention and introduce practical scheduling heuristics that temper const…

Crosslisted article(s) found for cs.IT. https://arxiv.org/list/cs.IT/new
[1/1]:
- Soft Graph Transformer for MIMO Detection
Jiadong Hong, Lei Liu, Xinyu Bian, Wenjie Wang, Zhaoyang Zhang

Hyperspectral data augmentation with transformer-based diffusion models
Mattia Ferrari, Lorenzo Bruzzone
https://arxiv.org/abs/2510.08363 https://arxiv.org…

Hyperspectral data augmentation with transformer-based diffusion models
The introduction of new generation hyperspectral satellite sensors, combined with advancements in deep learning methodologies, has significantly enhanced the ability to discriminate detailed land-cover classes at medium-large scales. However, a significant challenge in deep learning methods is the risk of overfitting when training networks with small labeled datasets. In this work, we propose a data augmentation technique that leverages a guided diffusion model. To effectively train the model w…

Lightweight Transformer for EEG Classification via Balanced Signed Graph Algorithm Unrolling
Junyi Yao, Parham Eftekhar, Gene Cheung, Xujin Chris Liu, Yao Wang, Wei Hu
https://arxiv.org/abs/2510.03027 …

Lightweight Transformer for EEG Classification via Balanced Signed Graph Algorithm Unrolling
Samples of brain signals collected by EEG sensors have inherent anti-correlations that are well modeled by negative edges in a finite graph. To differentiate epilepsy patients from healthy subjects using collected EEG signals, we build lightweight and interpretable transformer-like neural nets by unrolling a spectral denoising algorithm for signals on a balanced signed graph -- graph with no cycles of odd number of negative edges. A balanced signed graph has well-defined frequencies that map to…

BladderFormer: A Streaming Transformer for Real-Time Urological State Monitoring
Chengwei Zhou, Steve Majerus, Gourav Datta
https://arxiv.org/abs/2509.24178 https://

BladderFormer: A Streaming Transformer for Real-Time Urological State Monitoring
Bladder pressure monitoring systems are increasingly vital in diagnosing and managing urinary tract dysfunction. Existing solutions rely heavily on hand-crafted features and shallow classifiers, limiting their adaptability to complex signal dynamics. We propose a one-layer streaming transformer model for real-time classification of bladder pressure states, operating on wavelet-transformed representations of raw time-series data. Our model incorporates temporal multi-head self-attention and stat…

MAST: Multi-Agent Spatial Transformer for Learning to Collaborate
Damian Owerko, Frederic Vatnsdal, Saurav Agarwal, Vijay Kumar, Alejandro Ribeiro
https://arxiv.org/abs/2509.17195

MAST: Multi-Agent Spatial Transformer for Learning to Collaborate
This article presents a novel multi-agent spatial transformer (MAST) for learning communication policies in large-scale decentralized and collaborative multi-robot systems (DC-MRS). Challenges in collaboration in DC-MRS arise from: (i) partial observable states as robots make only localized perception, (ii) limited communication range with no central server, and (iii) independent execution of actions. The robots need to optimize a common task-specific objective, which, under the restricted sett…

TrackCore-F: Deploying Transformer-Based Subatomic Particle Tracking on FPGAs
Arjan Blankestijn, Uraz Odyurt, Amirreza Yousefzadeh
https://arxiv.org/abs/2509.26335 https://

TrackCore-F: Deploying Transformer-Based Subatomic Particle Tracking on FPGAs
The Transformer Machine Learning (ML) architecture has been gaining considerable momentum in recent years. In particular, computational High-Energy Physics tasks such as jet tagging and particle track reconstruction (tracking), have either achieved proper solutions, or reached considerable milestones using Transformers. On the other hand, the use of specialised hardware accelerators, especially FPGAs, is an effective method to achieve online, or pseudo-online latencies. The development and inte…

New Machine Learning Approaches for Intrusion Detection in ADS-B
Mika\"ela Ngambo\'e, Jean-Simon Marrocco, Jean-Yves Ouattara, Jos\'e M. Fernandez, Gabriela Nicolescu
https://arxiv.org/abs/2510.08333

New Machine Learning Approaches for Intrusion Detection in ADS-B
With the growing reliance on the vulnerable Automatic Dependent Surveillance-Broadcast (ADS-B) protocol in air traffic management (ATM), ensuring security is critical. This study investigates emerging machine learning models and training strategies to improve AI-based intrusion detection systems (IDS) for ADS-B. Focusing on ground-based ATM systems, we evaluate two deep learning IDS implementations: one using a transformer encoder and the other an extended Long Short-Term Memory (xLSTM) network…

Deconstructing Attention: Investigating Design Principles for Effective Language Modeling
Huiyin Xue, Nafise Sadat Moosavi, Nikolaos Aletras
https://arxiv.org/abs/2510.11602 htt…

Deconstructing Attention: Investigating Design Principles for Effective Language Modeling
The success of Transformer language models is widely credited to their dot-product attention mechanism, which interweaves a set of key design principles: mixing information across positions (enabling multi-token interactions), sequence-dependent activations (where attention weights adapt to each input), a specific mathematical form (dot-product similarities plus softmax weighting), and coupling of queries and keys to evolving hidden states (grounding attention in the current layer). However, th…

Replaced article(s) found for cs.CV. https://arxiv.org/list/cs.CV/new
[5/8]:
- Context Guided Transformer Entropy Modeling for Video Compression
Junlong Tong, Wei Zhang, Yaohui Jin, Xiaoyu Shen

Replaced article(s) found for cs.LG. https://arxiv.org/list/cs.LG/new
[9/14]:
- Hyper-STTN: Hypergraph Augmented Spatial-Temporal Transformer Network for Trajectory Prediction
Weizheng Wang, Baijian Yang, Sungeun Hong, Wenhai Sun, Byung-Cheol Min

Crosslisted article(s) found for eess.IV. https://arxiv.org/list/eess.IV/new
[1/1]:
- 3D Reconstruction from Transient Measurements with Time-Resolved Transformer
Yue Li, Shida Sun, Yu Hong, Feihu Xu, Zhiwei Xiong

Replaced article(s) found for cs.AI. https://arxiv.org/list/cs.AI/new
[2/9]:
- Spatial-Functional awareness Transformer-based graph archetype contrastive learning for Decoding ...
Yueming Sun, Long Yang

Lightweight and Data-Efficient MultivariateTime Series Forecasting using Residual-Stacked Gaussian (RS-GLinear) Architecture
Abukar Ali
https://arxiv.org/abs/2510.03788 https://…

Lightweight and Data-Efficient MultivariateTime Series Forecasting using Residual-Stacked Gaussian (RS-GLinear) Architecture
Following the success of Transformer architectures in language modeling, particularly their ability to capture long-range dependencies, researchers have explored how these architectures can be adapted for time-series forecasting. Transformer-based models have been proposed to handle both short- and long-term dependencies when predicting future values from historical data. However, studies such as those by Zeng et al. (2022) and Rizvi et al. (2025) have reported mixed results in long-term foreca…

IIET: Efficient Numerical Transformer via Implicit Iterative Euler Method
Xinyu Liu, Bei Li, Jiahao Liu, Junhao Ruan, Kechen Jiao, Hongyin Tang, Jingang Wang, Xiao Tong, Jingbo Zhu
https://arxiv.org/abs/2509.22463

IIET: Efficient Numerical Transformer via Implicit Iterative Euler Method
High-order numerical methods enhance Transformer performance in tasks like NLP and CV, but introduce a performance-efficiency trade-off due to increased computational overhead. Our analysis reveals that conventional efficiency techniques, such as distillation, can be detrimental to the performance of these models, exemplified by PCformer. To explore more optimizable ODE-based Transformer architectures, we propose the \textbf{I}terative \textbf{I}mplicit \textbf{E}uler \textbf{T}ransformer \text…

Utilizing dynamic sparsity on pretrained DETR
Reza Sedghi, Anand Subramoney, David Kappel
https://arxiv.org/abs/2510.09380 https://arxiv.org/pdf/2510.09380…

Utilizing dynamic sparsity on pretrained DETR
Efficient inference with transformer-based models remains a challenge, especially in vision tasks like object detection. We analyze the inherent sparsity in the MLP layers of DETR and introduce two methods to exploit it without retraining. First, we propose Static Indicator-Based Sparsification (SIBS), a heuristic method that predicts neuron inactivity based on fixed activation patterns. While simple, SIBS offers limited gains due to the input-dependent nature of sparsity. To address this, we i…

Towards fairer public transit: Real-time tensor-based multimodal fare evasion and fraud detection
Peter Wauyo, Dalia Bwiza, Alain Murara, Edwin Mugume, Eric Umuhoza
https://arxiv.org/abs/2510.02165

Towards fairer public transit: Real-time tensor-based multimodal fare evasion and fraud detection
This research introduces a multimodal system designed to detect fraud and fare evasion in public transportation by analyzing closed circuit television (CCTV) and audio data. The proposed solution uses the Vision Transformer for Video (ViViT) model for video feature extraction and the Audio Spectrogram Transformer (AST) for audio analysis. The system implements a Tensor Fusion Network (TFN) architecture that explicitly models unimodal and bimodal interactions through a 2-fold Cartesian product. …

Scattering Transformer: A Training-Free Transformer Architecture for Heart Murmur Detection
Rami Zewail
https://arxiv.org/abs/2509.18424 https://arxiv.org/…

TrackFormers Part 2: Enhanced Transformer-Based Models for High-Energy Physics Track Reconstruction
Sascha Caron, Nadezhda Dobreva, Maarten Kimpel, Uraz Odyurt, Slav Pshenov, Roberto Ruiz de Austri Bazan, Eugene Shalugin, Zef Wolffs, Yue Zhao
https://arxiv.org/abs/2509.26411

TrackFormers Part 2: Enhanced Transformer-Based Models for High-Energy Physics Track Reconstruction
High-Energy Physics experiments are rapidly escalating in generated data volume, a trend that will intensify with the upcoming High-Luminosity LHC upgrade. This surge in data necessitates critical revisions across the data processing pipeline, with particle track reconstruction being a prime candidate for improvement. In our previous work, we introduced "TrackFormers", a collection of Transformer-based one-shot encoder-only models that effectively associate hits with expected tracks. In this st…

Accent-Invariant Automatic Speech Recognition via Saliency-Driven Spectrogram Masking
Mohammad Hossein Sameti, Sepehr Harfi Moridani, Ali Zarean, Hossein Sameti
https://arxiv.org/abs/2510.09528

Accent-Invariant Automatic Speech Recognition via Saliency-Driven Spectrogram Masking
Pre-trained transformer-based models have significantly advanced automatic speech recognition (ASR), yet they remain sensitive to accent and dialectal variations, resulting in elevated word error rates (WER) in linguistically diverse languages such as English and Persian. To address this challenge, we propose an accent-invariant ASR framework that integrates accent and dialect classification into the recognition pipeline. Our approach involves training a spectrogram-based classifier to capture …

Transformer-Based Rate Prediction for Multi-Band Cellular Handsets
Ruibin Chen, Haozhe Lei, Hao Guo, Marco Mezzavilla, Hitesh Poddar, Tomoki Yoshimura, Sundeep Rangan
https://arxiv.org/abs/2509.25722

Transformer-Based Rate Prediction for Multi-Band Cellular Handsets
Cellular wireless systems are witnessing the proliferation of frequency bands over a wide spectrum, particularly with the expansion of new bands in FR3. These bands must be supported in user equipment (UE) handsets with multiple antennas in a constrained form factor. Rapid variations in channel quality across the bands from motion and hand blockage, limited field-of-view of antennas, and hardware and power-constrained measurement sparsity pose significant challenges to reliable multi-band chann…

Google debuts Titans, an architecture combining RNN speed with transformer performance for real-time learning, able to scale effectively to a 2M context window (Google Research)
https://research.google/blog/titans-miras-helping-ai-have-long-term-memory/

Titans + MIRAS: Helping AI have long-term memory

DiT-VTON: Diffusion Transformer Framework for Unified Multi-Category Virtual Try-On and Virtual Try-All with Integrated Image Editing
Qi Li, Shuwen Qiu, Julien Han, Xingzi Xu, Mehmet Saygin Seyfioglu, Kee Kiat Koo, Karim Bouyarmane
https://arxiv.org/abs/2510.04797

DiT-VTON: Diffusion Transformer Framework for Unified Multi-Category Virtual Try-On and Virtual Try-All with Integrated Image Editing
The rapid growth of e-commerce has intensified the demand for Virtual Try-On (VTO) technologies, enabling customers to realistically visualize products overlaid on their own images. Despite recent advances, existing VTO models face challenges with fine-grained detail preservation, robustness to real-world imagery, efficient sampling, image editing capabilities, and generalization across diverse product categories. In this paper, we present DiT-VTON, a novel VTO framework that leverages a Diffus…

A Hybrid DNN Transformer AE Framework for Corporate Tax Risk Supervision and Risk Level Assessment
Zhenzhen Song, Nanxi Wang, Hongji Li
https://arxiv.org/abs/2509.23862 https://…

A Hybrid DNN Transformer AE Framework for Corporate Tax Risk Supervision and Risk Level Assessment
Tax risk supervision has become a critical component of modern financial governance, as irregular tax behaviors and hidden compliance risks pose significant challenges to regulatory authorities and enterprises alike. Traditional rule-based methods often struggle to capture complex and dynamic tax-related anomalies in large-scale enterprise data. To address this issue, this paper proposes a hybrid deep learning framework (DNN-Transformer-Autoencoder) for corporate tax risk supervision and risk l…

AWARE, Beyond Sentence Boundaries: A Contextual Transformer Framework for Identifying Cultural Capital in STEM Narratives
Khalid Mehtab Khan, Anagha Kulkarni
https://arxiv.org/abs/2510.04983

AWARE, Beyond Sentence Boundaries: A Contextual Transformer Framework for Identifying Cultural Capital in STEM Narratives
Identifying cultural capital (CC) themes in student reflections can offer valuable insights that help foster equitable learning environments in classrooms. However, themes such as aspirational goals or family support are often woven into narratives, rather than appearing as direct keywords. This makes them difficult to detect for standard NLP models that process sentences in isolation. The core challenge stems from a lack of awareness, as standard models are pre-trained on general corpora, leav…

Pi-Transformer: A Physics-informed Attention Mechanism for Time Series Anomaly Detection
Sepehr Maleki, Negar Pourmoazemi
https://arxiv.org/abs/2509.19985 https://

Pi-Transformer: A Physics-informed Attention Mechanism for Time Series Anomaly Detection
Anomalies in multivariate time series often arise from temporal context and cross-channel coordination rather than isolated outliers. We present Pi-Transformer, a physics-informed transformer with two attention pathways: a data-driven series attention and a smoothly evolving prior attention that encodes temporal invariants such as scale-related self-similarity and phase synchrony. The prior acts as a stable reference that calibrates reconstruction error. During training, we pair a reconstructio…

GastroViT: A Vision Transformer Based Ensemble Learning Approach for Gastrointestinal Disease Classification with Grad CAM & SHAP Visualization
Sumaiya Tabassum, Md. Faysal Ahamed, Hafsa Binte Kibria, Md. Nahiduzzaman, Julfikar Haider, Muhammad E. H. Chowdhury, Mohammad Tariqul Islam
https://arxiv.org/abs/2509.26502

GastroViT: A Vision Transformer Based Ensemble Learning Approach for Gastrointestinal Disease Classification with Grad CAM & SHAP Visualization
The gastrointestinal (GI) tract of humans can have a wide variety of aberrant mucosal abnormality findings, ranging from mild irritations to extremely fatal illnesses. Prompt identification of gastrointestinal disorders greatly contributes to arresting the progression of the illness and improving therapeutic outcomes. This paper presents an ensemble of pre-trained vision transformers (ViTs) for accurately classifying endoscopic images of the GI tract to categorize gastrointestinal problems and …

The Potential of Second-Order Optimization for LLMs: A Study with Full Gauss-Newton
Natalie Abreu, Nikhil Vyas, Sham Kakade, Depen Morwani
https://arxiv.org/abs/2510.09378 https…

The Potential of Second-Order Optimization for LLMs: A Study with Full Gauss-Newton
Recent efforts to accelerate LLM pretraining have focused on computationally-efficient approximations that exploit second-order structure. This raises a key question for large-scale training: how much performance is forfeited by these approximations? To probe this question, we establish a practical upper bound on iteration complexity by applying full Gauss-Newton (GN) preconditioning to transformer models of up to 150M parameters. Our experiments show that full GN updates yield substantial gain…

Replaced article(s) found for cs.AI. https://arxiv.org/list/cs.AI/new
[6/13]:
- LIAM: Multimodal Transformer for Language Instructions, Images, Actions and Semantic Maps
Yihao Wang, Raphael Memmesheimer, Sven Behnke

Domain-Adapted Pre-trained Language Models for Implicit Information Extraction in Crash Narratives
Xixi Wang, Jordanka Kovaceva, Miguel Costa, Shuai Wang, Francisco Camara Pereira, Robert Thomson
https://arxiv.org/abs/2510.09434

Domain-Adapted Pre-trained Language Models for Implicit Information Extraction in Crash Narratives
Free-text crash narratives recorded in real-world crash databases have been shown to play a significant role in improving traffic safety. However, large-scale analyses remain difficult to implement as there are no documented tools that can batch process the unstructured, non standardized text content written by various authors with diverse experience and attention to detail. In recent years, Transformer-based pre-trained language models (PLMs), such as Bidirectional Encoder Representations from…

LucidFlux: Caption-Free Universal Image Restoration via a Large-Scale Diffusion Transformer
Song Fei, Tian Ye, Lujia Wang, Lei Zhu
https://arxiv.org/abs/2509.22414 https://

LucidFlux: Caption-Free Universal Image Restoration via a Large-Scale Diffusion Transformer
Universal image restoration (UIR) aims to recover images degraded by unknown mixtures while preserving semantics -- conditions under which discriminative restorers and UNet-based diffusion priors often oversmooth, hallucinate, or drift. We present LucidFlux, a caption-free UIR framework that adapts a large diffusion transformer (Flux.1) without image captions. LucidFlux introduces a lightweight dual-branch conditioner that injects signals from the degraded input and a lightly restored proxy to …

HRTFformer: A Spatially-Aware Transformer for Personalized HRTF Upsampling in Immersive Audio Rendering
Xuyi Hu, Jian Li, Shaojie Zhang, Stefan Goetz, Lorenzo Picinali, Ozgur B. Akan, Aidan O. T. Hogg
https://arxiv.org/abs/2510.01891

HRTFformer: A Spatially-Aware Transformer for Personalized HRTF Upsampling in Immersive Audio Rendering
Personalized Head-Related Transfer Functions (HRTFs) are starting to be introduced in many commercial immersive audio applications and are crucial for realistic spatial audio rendering. However, one of the main hesitations regarding their introduction is that creating personalized HRTFs is impractical at scale due to the complexities of the HRTF measurement process. To mitigate this drawback, HRTF spatial upsampling has been proposed with the aim of reducing measurements required. While prior w…

Augmented Intelligence, which is building neuro-symbolic AI models, raised $20M in a bridge SAFE round at a $750M valuation, bringing its total funding to ~$60M (Carl Franzen/VentureBeat)
https://venturebeat.com/ai/the-beginning-of-the-end…

Conv-like Scale-Fusion Time Series Transformer: A Multi-Scale Representation for Variable-Length Long Time Series
Kai Zhang, Siming Sun, Zhengyu Fan, Qinmin Yang, Xuejun Jiang
https://arxiv.org/abs/2509.17845

Conv-like Scale-Fusion Time Series Transformer: A Multi-Scale Representation for Variable-Length Long Time Series
Time series analysis faces significant challenges in handling variable-length data and achieving robust generalization. While Transformer-based models have advanced time series tasks, they often struggle with feature redundancy and limited generalization capabilities. Drawing inspiration from classical CNN architectures' pyramidal structure, we propose a Multi-Scale Representation Learning Framework based on a Conv-like ScaleFusion Transformer. Our approach introduces a temporal convolution-lik…

IBM releases Granite 4.0, an open source "enterprise-ready" LLM family with a hybrid architecture, claiming it uses significantly less RAM than traditional LLMs (Carl Franzen/VentureBeat)
https://venturebeat.com/ai/western-qwe…

Enhanced Arabic-language cyberbullying detection: deep embedding and transformer (BERT) approaches
Ebtesam Jaber Aljohani, Wael M. S. Yafoo
https://arxiv.org/abs/2510.02232 http…

Enhanced Arabic-language cyberbullying detection: deep embedding and transformer (BERT) approaches
Recent technological advances in smartphones and communications, including the growth of such online platforms as massive social media networks such as X (formerly known as Twitter) endangers young people and their emotional well-being by exposing them to cyberbullying, taunting, and bullying content. Most proposed approaches for automatically detecting cyberbullying have been developed around the English language, and methods for detecting Arabic-language cyberbullying are scarce. Methods for …

HART: Human Aligned Reconstruction Transformer
Xiyi Chen, Shaofei Wang, Marko Mihajlovic, Taewon Kang, Sergey Prokudin, Ming Lin
https://arxiv.org/abs/2509.26621 https://…

HART: Human Aligned Reconstruction Transformer
We introduce HART, a unified framework for sparse-view human reconstruction. Given a small set of uncalibrated RGB images of a person as input, it outputs a watertight clothed mesh, the aligned SMPL-X body mesh, and a Gaussian-splat representation for photorealistic novel-view rendering. Prior methods for clothed human reconstruction either optimize parametric templates, which overlook loose garments and human-object interactions, or train implicit functions under simplified camera assumptions,…

SINAI at eRisk@CLEF 2025: Transformer-Based and Conversational Strategies for Depression Detection
Alba Maria Marmol-Romero, Manuel Garcia-Vega, Miguel Angel Garcia-Cumbreras, Arturo Montejo-Raez
https://arxiv.org/abs/2509.19861

SINAI at eRisk@CLEF 2025: Transformer-Based and Conversational Strategies for Depression Detection
This paper describes the participation of the SINAI-UJA team in the eRisk@CLEF 2025 lab. Specifically, we addressed two of the proposed tasks: (i) Task 2: Contextualized Early Detection of Depression, and (ii) Pilot Task: Conversational Depression Detection via LLMs. Our approach for Task 2 combines an extensive preprocessing pipeline with the use of several transformer-based models, such as RoBERTa Base or MentalRoBERTA Large, to capture the contextual and sequential nature of multi-user conve…

Replaced article(s) found for cs.LG. https://arxiv.org/list/cs.LG/new
[6/8]:
- Rethinking Decoders for Transformer-based Semantic Segmentation: A Compression Perspective
Qishuai Wen, Chun-Guang Li

To Sink or Not to Sink: Visual Information Pathways in Large Vision-Language Models
Jiayun Luo, Wan-Cyuan Fan, Lyuyang Wang, Xiangteng He, Tanzila Rahman, Purang Abolmaesumi, Leonid Sigal
https://arxiv.org/abs/2510.08510

To Sink or Not to Sink: Visual Information Pathways in Large Vision-Language Models
Large Vision Language Models (LVLMs) have recently emerged as powerful architectures capable of understanding and reasoning over both visual and textual information. These models typically rely on two key components: a Vision Transformer (ViT) and a Large Language Model (LLM). ViT encodes visual content into a sequence of image tokens and serves as the perceptual front-end -- the eyes of the model. In contrast, the LLM interprets these tokens to perform high-level reasoning, generates responses…

Transformer-Encoder Trees for Efficient Multilingual Machine Translation and Speech Translation
Yiwen Guan, Jacob Whitehill
https://arxiv.org/abs/2509.17930 https://

Transformer-Encoder Trees for Efficient Multilingual Machine Translation and Speech Translation
Multilingual translation faces challenges of computational redundancy and limited accuracy for low-resource languages, especially in speech translation. To address this, we propose a novel hierarchical Transformer Encoder Tree (TET) combined with non-autoregressive encoder-only models trained with Connectionist Temporal Classification for multilingual translation. By sharing intermediate representations among linguistically similar target languages, TET can improve accuracy on low-resource lang…

GraDeT-HTR: A Resource-Efficient Bengali Handwritten Text Recognition System utilizing Grapheme-based Tokenizer and Decoder-only Transformer
Md. Mahmudul Hasan, Ahmed Nesar Tahsin Choudhury, Mahmudul Hasan, Md. Mosaddek Khan
https://arxiv.org/abs/2509.18081

GraDeT-HTR: A Resource-Efficient Bengali Handwritten Text Recognition System utilizing Grapheme-based Tokenizer and Decoder-only Transformer
Despite Bengali being the sixth most spoken language in the world, handwritten text recognition (HTR) systems for Bengali remain severely underdeveloped. The complexity of Bengali script--featuring conjuncts, diacritics, and highly variable handwriting styles--combined with a scarcity of annotated datasets makes this task particularly challenging. We present GraDeT-HTR, a resource-efficient Bengali handwritten text recognition system based on a Grapheme-aware Decoder-only Transformer architectu…

Grouped Differential Attention
Junghwan Lim, Sungmin Lee, Dongseok Kim, Wai Ting Cheung, Beomgyu Kim, Taehwan Kim, Haesol Lee, Junhyeok Lee, Dongpin Oh, Eunhwan Park
https://arxiv.org/abs/2510.06949

Grouped Differential Attention
The self-attention mechanism, while foundational to modern Transformer architectures, suffers from a critical inefficiency: it frequently allocates substantial attention to redundant or noisy context. Differential Attention addressed this by using subtractive attention maps for signal and noise, but its required balanced head allocation imposes rigid constraints on representational flexibility and scalability. To overcome this, we propose Grouped Differential Attention (GDA), a novel approach…

AI-Driven Radiology Report Generation for Traumatic Brain Injuries
Riadh Bouslimi, Houda Trabelsi, Wahiba Ben Abdssalem Karaa, Hana Hedhli
https://arxiv.org/abs/2510.08498 https…

AI-Driven Radiology Report Generation for Traumatic Brain Injuries
Traumatic brain injuries present significant diagnostic challenges in emergency medicine, where the timely interpretation of medical images is crucial for patient outcomes. In this paper, we propose a novel AI-based approach for automatic radiology report generation tailored to cranial trauma cases. Our model integrates an AC-BiFPN with a Transformer architecture to capture and process complex medical imaging data such as CT and MRI scans. The AC-BiFPN extracts multi-scale features, enabling th…

ELMUR: External Layer Memory with Update/Rewrite for Long-Horizon RL
Egor Cherepanov, Alexey K. Kovalev, Aleksandr I. Panov
https://arxiv.org/abs/2510.07151 https://

ELMUR: External Layer Memory with Update/Rewrite for Long-Horizon RL
Real-world robotic agents must act under partial observability and long horizons, where key cues may appear long before they affect decision making. However, most modern approaches rely solely on instantaneous information, without incorporating insights from the past. Standard recurrent or transformer models struggle with retaining and leveraging long-term dependencies: context windows truncate history, while naive memory extensions fail under scale and sparsity. We propose ELMUR (External Laye…

Heptapod: Language Modeling on Visual Signals
Yongxin Zhu, Jiawei Chen, Yuanzhe Chen, Zhuo Chen, Dongya Jia, Jian Cong, Xiaobin Zhuang, Yuping Wang, Yuxuan Wang
https://arxiv.org/abs/2510.06673

Heptapod: Language Modeling on Visual Signals
We introduce Heptapod, an image autoregressive model that adheres to the foundational principles of language modeling. Heptapod employs \textbf{causal attention}, \textbf{eliminates reliance on CFG}, and \textbf{eschews the trend of semantic tokenizers}. Our key innovation is \textit{next 2D distribution prediction}: a causal Transformer with reconstruction-focused visual tokenizer, learns to predict the distribution over the entire 2D spatial grid of images at each timestep. This learning obje…

Bayesian Modelling of Multi-Year Crop Type Classification Using Deep Neural Networks and Hidden Markov Models
Gianmarco Perantoni, Giulio Weikmann, Lorenzo Bruzzone
https://arxiv.org/abs/2510.07008

Bayesian Modelling of Multi-Year Crop Type Classification Using Deep Neural Networks and Hidden Markov Models
The temporal consistency of yearly land-cover maps is of great importance to model the evolution and change of the land cover over the years. In this paper, we focus the attention on a novel approach to classification of yearly satellite image time series (SITS) that combines deep learning with Bayesian modelling, using Hidden Markov Models (HMMs) integrated with Transformer Encoder (TE) based DNNs. The proposed approach aims to capture both i) intricate temporal correlations in yearly SITS and…

Optimizing Inference in Transformer-Based Models: A Multi-Method Benchmark
Siu Hang Ho, Prasad Ganesan, Nguyen Duong, Daniel Schlabig
https://arxiv.org/abs/2509.17894 https://…

Optimizing Inference in Transformer-Based Models: A Multi-Method Benchmark
Efficient inference is a critical challenge in deep generative modeling, particularly as diffusion models grow in capacity and complexity. While increased complexity often improves accuracy, it raises compute costs, latency, and memory requirements. This work investigates techniques such as pruning, quantization, knowledge distillation, and simplified attention to reduce computational overhead without impacting performance. The study also explores the Mixture of Experts (MoE) approach to furthe…

Quantized Visual Geometry Grounded Transformer
Weilun Feng, Haotong Qin, Mingqiang Wu, Chuanguang Yang, Yuqi Li, Xiangqi Li, Zhulin An, Libo Huang, Yulun Zhang, Michele Magno, Yongjun Xu
https://arxiv.org/abs/2509.21302

Quantized Visual Geometry Grounded Transformer
Learning-based 3D reconstruction models, represented by Visual Geometry Grounded Transformers (VGGTs), have made remarkable progress with the use of large-scale transformers. Their prohibitive computational and memory costs severely hinder real-world deployment. Post-Training Quantization (PTQ) has become a common practice for compressing and accelerating models. However, we empirically observe that PTQ faces unique obstacles when compressing billion-scale VGGTs: the data-independent special to…

Lung Infection Severity Prediction Using Transformers with Conditional TransMix Augmentation and Cross-Attention
Bouthaina Slika, Fadi Dornaika, Fares Bougourzi, Karim Hammoudi
https://arxiv.org/abs/2510.06887

Lung Infection Severity Prediction Using Transformers with Conditional TransMix Augmentation and Cross-Attention
Lung infections, particularly pneumonia, pose serious health risks that can escalate rapidly, especially during pandemics. Accurate AI-based severity prediction from medical imaging is essential to support timely clinical decisions and optimize patient outcomes. In this work, we present a novel method applicable to both CT scans and chest X-rays for assessing lung infection severity. Our contributions are twofold: (i) QCross-Att-PVT, a Transformer-based architecture that integrates parallel enc…

Crosslisted article(s) found for cs.LG. https://arxiv.org/list/cs.LG/new
[1/8]:
- ReplaceMe: Network Simplification via Depth Pruning and Transformer Block Linearization
Shopkhoev, Ali, Zhussip, Malykh, Lefkimmiatis, Komodakis, Zagoruyko

Enhancing Transformer-Based Vision Models: Addressing Feature Map Anomalies Through Novel Optimization Strategies
Sumit Mamtani
https://arxiv.org/abs/2509.19687 https://

Enhancing Transformer-Based Vision Models: Addressing Feature Map Anomalies Through Novel Optimization Strategies
Vision Transformers (ViTs) have demonstrated superior performance across a wide range of computer vision tasks. However, structured noise artifacts in their feature maps hinder downstream applications such as segmentation and depth estimation. We propose two novel and lightweight optimisation techniques- Structured Token Augmentation (STA) and Adaptive Noise Filtering (ANF)- to improve interpretability and mitigate these artefacts. STA enhances token diversity through spatial perturbations duri…

PS3: A Multimodal Transformer Integrating Pathology Reports with Histology Images and Biological Pathways for Cancer Survival Prediction
Manahil Raza, Ayesha Azam, Talha Qaiser, Nasir Rajpoot
https://arxiv.org/abs/2509.20022

PS3: A Multimodal Transformer Integrating Pathology Reports with Histology Images and Biological Pathways for Cancer Survival Prediction
Current multimodal fusion approaches in computational oncology primarily focus on integrating multi-gigapixel histology whole slide images (WSIs) with genomic or transcriptomic data, demonstrating improved survival prediction. We hypothesize that incorporating pathology reports can further enhance prognostic performance. Pathology reports, as essential components of clinical workflows, offer readily available complementary information by summarizing histopathological findings and integrating ex…

EfficienT-HDR: An Efficient Transformer-Based Framework via Multi-Exposure Fusion for HDR Reconstruction
Yu-Shen Huang, Tzu-Han Chen, Cheng-Yen Hsiao, Shaou-Gang Miaou
https://arxiv.org/abs/2509.19779 …

EfficienT-HDR: An Efficient Transformer-Based Framework via Multi-Exposure Fusion for HDR Reconstruction
Achieving high-quality High Dynamic Range (HDR) imaging on resource-constrained edge devices is a critical challenge in computer vision, as its performance directly impacts downstream tasks such as intelligent surveillance and autonomous driving. Multi-Exposure Fusion (MEF) is a mainstream technique to achieve this goal; however, existing methods generally face the dual bottlenecks of high computational costs and ghosting artifacts, hindering their widespread deployment. To this end, this study…

Visual Representations inside the Language Model
Benlin Liu, Amita Kamath, Madeleine Grunde-McLaughlin, Winson Han, Ranjay Krishna
https://arxiv.org/abs/2510.04819 https://

Visual Representations inside the Language Model
Despite interpretability work analyzing VIT encoders and transformer activations, we don't yet understand why Multimodal Language Models (MLMs) struggle on perception-heavy tasks. We offer an under-studied perspective by examining how popular MLMs (LLaVA-OneVision, Qwen2.5-VL, and Llama-3-LLaVA-NeXT) process their visual key-value tokens. We first study the flow of visual information through the language model, finding that image value tokens encode sufficient information to perform several per…