Tootfinder

Native Hybrid Attention for Efficient Sequence Modeling
Jusen Du, Jiaxi Hu, Tao Zhang, Weigao Sun, Yu Cheng
https://arxiv.org/abs/2510.07019 https://arxiv.…

Native Hybrid Attention for Efficient Sequence Modeling
Transformers excel at sequence modeling but face quadratic complexity, while linear attention offers improved efficiency but often compromises recall accuracy over long contexts. In this work, we introduce Native Hybrid Attention (NHA), a novel hybrid architecture of linear and full attention that integrates both intra \& inter-layer hybridization into a unified layer design. NHA maintains long-term context in key-value slots updated by a linear RNN, and augments them with short-term tokens fro…

Neurocognitive Modeling for Text Generation: Deep Learning Architecture for EEG Data
Khushiyant
https://arxiv.org/abs/2509.07202 https://arxiv.org/pdf/2509…

Neurocognitive Modeling for Text Generation: Deep Learning Architecture for EEG Data
Text generating capabilities have undergone a substantial transformation with the introduction of large language models (LLMs). Electroencephalography (EEG)-based text production is still difficult, though, because it requires a lot of data and processing power. This paper introduces a new method that combines the use of the Gemma 2B LLM with a classifier-LLM architecture to incorporate a Recurrent Neural Network (RNN) encoder. Our approach drastically lowers the amount of data and compute powe…

Identifying Exoplanets with Deep Learning: A CNN and RNN Classifier for Kepler DR25 and Candidate Vetting
Bibin Thomas, Vittal Bhat M, Salman Arafath Mohammed, Abdul Wase Mohammed, Adis Abebaw Dessalegn, Mohit Mittal
https://arxiv.org/abs/2509.04793

Identifying Exoplanets with Deep Learning: A CNN and RNN Classifier for Kepler DR25 and Candidate Vetting
The rapid expansion of exoplanet survey missions such as Kepler, TESS, and the upcoming PLATO mission has generated massive light-curve datasets that challenge traditional vetting pipelines. We introduce a hybrid deep-learning framework that integrates convolutional networks, bidirectional LSTMs, and an attention mechanism to identify planetary transit signals with improved accuracy and interpretability. Trained on Kepler DR25 data, the model achieves F1 = $0.910 \pm 0.008$ (AUC--ROC = $0.984 \…

Dissecting Larval Zebrafish Hunting using Deep Reinforcement Learning Trained RNN Agents
Raaghav Malik, Satpreet H. Singh, Sonja Johnson-Yu, Nathan Wu, Roy Harpaz, Florian Engert, Kanaka Rajan
https://arxiv.org/abs/2510.03699

Dissecting Larval Zebrafish Hunting using Deep Reinforcement Learning Trained RNN Agents
Larval zebrafish hunting provides a tractable setting to study how ecological and energetic constraints shape adaptive behavior in both biological brains and artificial agents. Here we develop a minimal agent-based model, training recurrent policies with deep reinforcement learning in a bout-based zebrafish simulator. Despite its simplicity, the model reproduces hallmark hunting behaviors -- including eye vergence-linked pursuit, speed modulation, and stereotyped approach trajectories -- that c…

GRNND: A GPU-Parallel Relative NN-Descent Algorithm for Efficient Approximate Nearest Neighbor Graph Construction
Xiang Li (Nanjing University), Qiong Chang (Institute of Science Tokyo), Yun Li (Nanjing University), Jun Miyazaki (Institute of Science Tokyo)
https://arxiv.org/abs/2510.02774

GRNND: A GPU-Parallel Relative NN-Descent Algorithm for Efficient Approximate Nearest Neighbor Graph Construction
Relative Nearest Neighbor Descent (RNN-Descent) is a state-of-the-art algorithm for constructing sparse approximate nearest neighbor (ANN) graphs by combining the iterative refinement of NN-Descent with the edge-pruning rules of the Relative Neighborhood Graph (RNG). It has demonstrated strong effectiveness in large-scale search tasks such as information retrieval and related tasks. However, as the amount and dimensionality of data increase, the complexity of graph construction in RNN-Descent r…

StateX: Enhancing RNN Recall via Post-training State Expansion
Xingyu Shen, Yingfa Chen, Zhen Leng Thai, Xu Han, Zhiyuan Liu, Maosong Sun
https://arxiv.org/abs/2509.22630 https:…

StateX: Enhancing RNN Recall via Post-training State Expansion
While Transformer-based models have demonstrated remarkable language modeling performance, their high complexities result in high costs when processing long contexts. In contrast, recurrent neural networks (RNNs) such as linear attention and state space models have gained popularity due to their constant per-token complexities. However, these recurrent models struggle with tasks that require accurate recall of contextual information from long contexts, because all contextual information is comp…

Is attention truly all we need? An empirical study of asset pricing in pretrained RNN sparse and global attention models
Shanyan Lai
https://arxiv.org/abs/2508.19006 https://

Is attention truly all we need? An empirical study of asset pricing in pretrained RNN sparse and global attention models
This study investigates the pretrained RNN attention models with the mainstream attention mechanisms such as additive attention, Luong's three attentions, global self-attention (Self-att) and sliding window sparse attention (Sparse-att) for the empirical asset pricing research on top 420 large-cap US stocks. This is the first paper on the large-scale state-of-the-art (SOTA) attention mechanisms applied in the asset pricing context. They overcome the limitations of the traditional machine learni…

Lipschitz-Based Robustness Certification for Recurrent Neural Networks via Convex Relaxation
Paul Hamelbeck, Johannes Schiffer
https://arxiv.org/abs/2509.17898 https://

Lipschitz-Based Robustness Certification for Recurrent Neural Networks via Convex Relaxation
Robustness certification against bounded input noise or adversarial perturbations is increasingly important for deployment recurrent neural networks (RNNs) in safety-critical control applications. To address this challenge, we present RNN-SDP, a relaxation based method that models the RNN's layer interactions as a convex problem and computes a certified upper bound on the Lipschitz constant via semidefinite programming (SDP). We also explore an extension that incorporates known input constraint…

ELIXIR: Efficient and LIghtweight model for eXplaIning Recommendations
Ben Kabongo, Vincent Guigue, Pirmin Lemberger
https://arxiv.org/abs/2508.20312 https://

ELIXIR: Efficient and LIghtweight model for eXplaIning Recommendations
Collaborative filtering drives many successful recommender systems but struggles with fine-grained user-item interactions and explainability. As users increasingly seek transparent recommendations, generating textual explanations through language models has become a critical research area. Existing methods employ either RNNs or Transformers. However, RNN-based approaches fail to leverage the capabilities of pre-trained Transformer models, whereas Transformer-based methods often suffer from subo…

An Adversarial Robust Behavior Sequence Anomaly Detection Approach Based on Critical Behavior Unit Learning
Dongyang Zhan, Kai Tan, Lin Ye, Xiangzhan Yu, Hongli Zhang, Zheng He
https://arxiv.org/abs/2509.15756

An Adversarial Robust Behavior Sequence Anomaly Detection Approach Based on Critical Behavior Unit Learning
Sequential deep learning models (e.g., RNN and LSTM) can learn the sequence features of software behaviors, such as API or syscall sequences. However, recent studies have shown that these deep learning-based approaches are vulnerable to adversarial samples. Attackers can use adversarial samples to change the sequential characteristics of behavior sequences and mislead malware classifiers. In this paper, an adversarial robustness anomaly detection method based on the analysis of behavior units i…

Overcoming the curse of dimensionality: Enabling multi-layer photon transport with recurrent neural network
Daniel Carne, Ziqi Guo, Xiulin Ruan
https://arxiv.org/abs/2509.22890 …

Overcoming the curse of dimensionality: Enabling multi-layer photon transport with recurrent neural network
Monte Carlo simulations are commonly used to calculate photon reflectance, absorptance, and transmittance of multi-layer scattering and absorbing media, but they can quickly become prohibitively expensive as the number of layers increases. In this study, we show that although a plain neural network suffers from the curse of dimensionality and fails to yield acceptable predictions of multilayer media, we introduce a recurrent neural network (RNN) trained on the same Monte Carlo simulation datase…

MuSpike: A Benchmark and Evaluation Framework for Symbolic Music Generation with Spiking Neural Networks
Qian Liang, Menghaoran Tang, Yi Zeng
https://arxiv.org/abs/2508.19251 ht…

MuSpike: A Benchmark and Evaluation Framework for Symbolic Music Generation with Spiking Neural Networks
Symbolic music generation has seen rapid progress with artificial neural networks, yet remains underexplored in the biologically plausible domain of spiking neural networks (SNNs), where both standardized benchmarks and comprehensive evaluation methods are lacking. To address this gap, we introduce MuSpike, a unified benchmark and evaluation framework that systematically assesses five representative SNN architectures (SNN-CNN, SNN-RNN, SNN-LSTM, SNN-GAN and SNN-Transformer) across five typical …

CKFNet: Neural Network Aided Cubature Kalman filtering
Jinhui Hu, Haiquan Zhao, Yi Peng
https://arxiv.org/abs/2508.09727 https://arxiv.org/pdf/2508.09727…

CKFNet: Neural Network Aided Cubature Kalman filtering
The cubature Kalman filter (CKF), while theoretically rigorous for nonlinear estimation, often suffers performance degradation due to model-environment mismatches in practice. To address this limitation, we propose CKFNet-a hybrid architecture that synergistically integrates recurrent neural networks (RNN) with the CKF framework while preserving its cubature principles. Unlike conventional model-driven approaches, CKFNet embeds RNN modules in the prediction phase to dynamically adapt to unmodel…

Stacked Hybrid RNN-CNN Reconstruction of X-ray Influence on 21-cm Brightness Temperature
S. Mobina Hosseini, Bahareh Soleimanpour Salmasi
https://arxiv.org/abs/2508.05842 https:…

Stacked Hybrid RNN-CNN Reconstruction of X-ray Influence on 21-cm Brightness Temperature
The X-ray photons substantially affect the thermal and ionization states of the intergalactic medium (IGM) during the Epoch of Reionization (EoR), thereby significantly influencing the 21-cm line observables such as its sky-averaged (global) brightness temperature. Nevertheless, the complicated dependency of astrophysical processes on a broad spectrum of parameters, including X-ray efficiency, spectral characteristics, and gas dynamics, makes precisely simulating the effect of X-ray flux challe…

Response function as a quantitative measure of consciousness in brain dynamics
Wenkang Du, Haiping Huang
https://arxiv.org/abs/2509.00730 https://arxiv.org…

Response function as a quantitative measure of consciousness in brain dynamics
Understanding the neural correlates of consciousness remains a central challenge in neuroscience. In this study, we investigate the relationship between consciousness and neural responsiveness by analyzing intracranial ECoG recordings from non-human primates across three distinct states: wakefulness, anesthesia, and recovery. Using a nonequilibrium recurrent neural network (RNN) model, we fit state-dependent cortical dynamics to extract the neural response function as a dynamics complexity indi…

Crosslisted article(s) found for econ.EM. https://arxiv.org/list/econ.EM/new
[1/1]:
- Is attention truly all we need? An empirical study of asset pricing in pretrained RNN sparse and ...
Shanyan Lai

Toward Practical Equilibrium Propagation: Brain-inspired Recurrent Neural Network with Feedback Regulation and Residual Connections
Zhuo Liu, Tao Chen
https://arxiv.org/abs/2508.11659

Toward Practical Equilibrium Propagation: Brain-inspired Recurrent Neural Network with Feedback Regulation and Residual Connections
Brain-like intelligent systems need brain-like learning methods. Equilibrium Propagation (EP) is a biologically plausible learning framework with strong potential for brain-inspired computing hardware. However, existing im-plementations of EP suffer from instability and prohibi-tively high computational costs. Inspired by the structure and dynamics of the brain, we propose a biologically plau-sible Feedback-regulated REsidual recurrent neural network (FRE-RNN) and study its learning performance…

Investment Portfolio Optimization Based on Modern Portfolio Theory and Deep Learning Models
Maciej Wysocki, Pawe{\l} Sakowski
https://arxiv.org/abs/2508.14999 https://

Investment Portfolio Optimization Based on Modern Portfolio Theory and Deep Learning Models
This paper investigates an important problem of an appropriate variance-covariance matrix estimation in the Modern Portfolio Theory. We propose a novel framework for variancecovariance matrix estimation for purposes of the portfolio optimization, which is based on deep learning models. We employ the long short-term memory (LSTM) recurrent neural networks (RNN) along with two probabilistic deep learning models: DeepVAR and GPVAR to the task of one-day ahead multivariate forecasting. We then use …

Optimised neural networks for online processing of ATLAS calorimeter data on FPGAs
Georges Aad, Raphael Bertrand, Lauri Laatu, Emmanuel Monnier, Arno Straessner, Nairit Sur, Johann C. Voigt
https://arxiv.org/abs/2510.11469

Optimised neural networks for online processing of ATLAS calorimeter data on FPGAs
A study of neural network architectures for the reconstruction of the energy deposited in the cells of the ATLAS liquid-argon calorimeters under high pile-up conditions expected at the HL-LHC is presented. These networks are designed to run on the FPGA-based readout hardware of the calorimeters under strict size and latency constraints. Several architectures, including Dense, Recurrent (RNN), and Convolutional (CNN) neural networks, are optimised using a Bayesian procedure that balances energy …

The Digital Sous Chef -- A Comparative Study on Fine-Tuning Language Models for Recipe Generation
Shubham Pundhir, Ganesh Bagler
https://arxiv.org/abs/2508.14718 https://…

The Digital Sous Chef -- A Comparative Study on Fine-Tuning Language Models for Recipe Generation
We established a rigorous benchmark for text-based recipe generation, a fundamental task in natural language generation. We present a comprehensive comparative study contrasting a fine-tuned GPT-2 large (774M) model against the GPT-2 small (124M) model and traditional LSTM/RNN baselines on the 5-cuisine corpus from RecipeDB. Our key contribution is a targeted tokenization strategy that augments the vocabulary with 23 common fraction tokens and custom structural markers. This approach addresses …

Overcoming Latency Bottlenecks in On-Device Speech Translation: A Cascaded Approach with Alignment-Based Streaming MT
Zeeshan Ahmed, Frank Seide, Niko Moritz, Ju Lin, Ruiming Xie, Simone Merello, Zhe Liu, Christian Fuegen
https://arxiv.org/abs/2508.13358

Overcoming Latency Bottlenecks in On-Device Speech Translation: A Cascaded Approach with Alignment-Based Streaming MT
This paper tackles several challenges that arise when integrating Automatic Speech Recognition (ASR) and Machine Translation (MT) for real-time, on-device streaming speech translation. Although state-of-the-art ASR systems based on Recurrent Neural Network Transducers (RNN-T) can perform real-time transcription, achieving streaming translation in real-time remains a significant challenge. To address this issue, we propose a simultaneous translation approach that effectively balances translation…