Tootfinder

GDNSQ: Gradual Differentiable Noise Scale Quantization for Low-bit Neural Networks
Sergey Salishev, Ian Akhremchik
https://arxiv.org/abs/2508.14004 https://

GDNSQ: Gradual Differentiable Noise Scale Quantization for Low-bit Neural Networks
Quantized neural networks can be viewed as a chain of noisy channels, where rounding in each layer reduces capacity as bit-width shrinks; the floating-point (FP) checkpoint sets the maximum input rate. We track capacity dynamics as the average bit-width decreases and identify resulting quantization bottlenecks by casting fine-tuning as a smooth, constrained optimization problem. Our approach employs a fully differentiable Straight-Through Estimator (STE) with learnable bit-width, noise scale an…

Fully Spiking Actor-Critic Neural Network for Robotic Manipulation
Liwen Zhang, Heng Deng, Guanghui Sun
https://arxiv.org/abs/2508.12038 https://arxiv.org/…

Fully Spiking Actor-Critic Neural Network for Robotic Manipulation
This study proposes a hybrid curriculum reinforcement learning (CRL) framework based on a fully spiking neural network (SNN) for 9-degree-of-freedom robotic arms performing target reaching and grasping tasks. To reduce network complexity and inference latency, the SNN architecture is simplified to include only an input and an output layer, which shows strong potential for resource-constrained environments. Building on the advantages of SNNs-high inference speed, low energy consumption, and spik…

Synchronization and semantization in deep spiking networks
Jonas Oberste-Frielinghaus, Anno C. Kurth, Julian G\"oltz, Laura Kriener, Junji Ito, Mihai A. Petrovici, Sonja Gr\"un
https://arxiv.org/abs/2508.12975

Synchronization and semantization in deep spiking networks
Recent studies have shown how spiking networks can learn complex functionality through error-correcting plasticity, but the resulting structures and dynamics remain poorly studied. To elucidate how these models may link to observed dynamics in vivo and thus how they may ultimately explain cortical computation, we need a better understanding of their emerging patterns. We train a multi-layer spiking network, as a conceptual analog of the bottom-up visual hierarchy, for visual input classificatio…

Beyond Semantic Understanding: Preserving Collaborative Frequency Components in LLM-based Recommendation
Minhao Wang, Yunhang He, Cong Xu, Zhangchi Zhu, Wei Zhang
https://arxiv.org/abs/2508.10312

Beyond Semantic Understanding: Preserving Collaborative Frequency Components in LLM-based Recommendation
Recommender systems in concert with Large Language Models (LLMs) present promising avenues for generating semantically-informed recommendations. However, LLM-based recommenders exhibit a tendency to overemphasize semantic correlations within users' interaction history. When taking pretrained collaborative ID embeddings as input, LLM-based recommenders progressively weaken the inherent collaborative signals as the embeddings propagate through LLM backbones layer by layer, as opposed to tradition…

Input Conditioned Layer Dropping in Speech Foundation Models
Abdul Hannan, Daniele Falavigna, Alessio Brutti
https://arxiv.org/abs/2507.07954 https://

Input Conditioned Layer Dropping in Speech Foundation Models
Curating foundation speech models for edge and IoT settings, where computational resources vary over time, requires dynamic architectures featuring adaptable reduction strategies. One emerging approach is layer dropping ($\mathcal{LD}$) which skips fraction of the layers of a backbone network during inference to reduce the computational load. This allows transforming static models into dynamic ones. However, existing approaches exhibit limitations either in the mode of selecting layers or by si…

Optimizing Fluid Antenna Configurations for Constructive Interference Precoding
Wenxuan Sun, Mingjie Shao, Luteng Zhu, Yao Ge, Tong Zhang, Zhi Liu
https://arxiv.org/abs/2507.11093

Optimizing Fluid Antenna Configurations for Constructive Interference Precoding
The fluid antenna system (FAS) has emerged as a new physical-layer concept to provide enhanced propagation conditions for multiuser multiple-input multiple-output (MIMO) communications over conventional fixed arrays. This work focuses on minimizing the maximum symbol error probability (SEP) under $M$-ary phase shift keying (MPSK) signaling in a multiuser downlink equipped with FAS, where each antenna moves within nonoverlapping intervals. This specific problem of joint SEP minimization with FAS…

Error Exponents for Quantum Packing Problems via An Operator Layer Cake Theorem
Hao-Chung Cheng, Po-Chieh Liu
https://arxiv.org/abs/2507.06232 https://

Error Exponents for Quantum Packing Problems via An Operator Layer Cake Theorem
In this work, we prove a one-shot random coding bound for classical-quantum channel coding, a problem conjectured by Burnashev and Holevo in 1998. By choosing the optimal input distribution, we recover the optimal error exponent (i.e., the reliability function) of classical-quantum channels for rates above the critical rate. Our result extends to various quantum packing-type problems, including classical communication over any fully quantum channel with or without entanglement-assistance, const…

What One Cannot, Two Can: Two-Layer Transformers Provably Represent Induction Heads on Any-Order Markov Chains
Chanakya Ekbote, Marco Bondaschi, Nived Rajaraman, Jason D. Lee, Michael Gastpar, Ashok Vardhan Makkuva, Paul Pu Liang
https://arxiv.org/abs/2508.07208

What One Cannot, Two Can: Two-Layer Transformers Provably Represent Induction Heads on Any-Order Markov Chains
In-context learning (ICL) is a hallmark capability of transformers, through which trained models learn to adapt to new tasks by leveraging information from the input context. Prior work has shown that ICL emerges in transformers due to the presence of special circuits called induction heads. Given the equivalence between induction heads and conditional k-grams, a recent line of work modeling sequential inputs as Markov processes has revealed the fundamental impact of model depth on its ICL capa…

Fast Forward and Inverse Thermal Modeling for Parameter Estimation of Multi-Layer Composites -- Part II: Inverse Modeling and Applications
Gan Fu, Mitrofan Curti, Calina Ciuhu, Elena A. Lomonova
https://arxiv.org/abs/2507.06746

Fast Forward and Inverse Thermal Modeling for Parameter Estimation of Multi-Layer Composites -- Part II: Inverse Modeling and Applications
A fast inverse heat conduction model (IHCM) is developed for estimating unknown properties of multi-layer composites considering internal heat generation. This work builds on the validated analytical forward models presented in Part I. Transient temperature at a single point is used as input, with the objective function minimized through an interior-point optimization algorithm. The IHCM accurately estimates thermal properties such as thermal conductivity, specific heat capacity, density, and h…

Ken Utilization Layer: Hebbian Replay Within a Student's Ken for Adaptive Knowledge Tracing
Grey Kuling, Marinka Zitnik
https://arxiv.org/abs/2507.00032

Ken Utilization Layer: Hebbian Replay Within a Student's Ken for Adaptive Knowledge Tracing
We introduce KUL-KT, a biologically inspired architecture for knowledge tracing (KT), combining Hebbian memory encoding with gradient-based consolidation in a scalable, input-agnostic framework. KUL-KT adapts the principle of memory consolidation in neural systems, to student modeling by introducing two key innovations: (i) a time-decaying Hebbian memory update that enables graceful forgetting, and (ii) a novel Loss-aligned Internal Target (LIT) method to compute an ideal internal state, allowi…

Implementing Neural Networks Over-the-Air via Reconfigurable Intelligent Surfaces
Meng Hua, Chenghong Bian, Haotian Wu, Deniz Gunduz
https://arxiv.org/abs/2508.01840 https://

Implementing Neural Networks Over-the-Air via Reconfigurable Intelligent Surfaces
In this paper, we investigate reconfigurable intelligent surface (RIS)-aided multiple-input-multiple-output (MIMO) OAC systems designed to emulate the fully-connected (FC) layer of a neural network (NN) via analog OAC, where the RIS and the transceivers are jointly adjusted to engineer the ambient wireless propagation environment to emulate the weights of the target FC layer. We refer to this novel computational paradigm as AirFC. We first study the case in which the precoder, combiner, and RIS…

AttZoom: Attention Zoom for Better Visual Features
Daniel DeAlcala, Aythami Morales, Julian Fierrez, Ruben Tolosana
https://arxiv.org/abs/2508.03625 https://

AttZoom: Attention Zoom for Better Visual Features
We present Attention Zoom, a modular and model-agnostic spatial attention mechanism designed to improve feature extraction in convolutional neural networks (CNNs). Unlike traditional attention approaches that require architecture-specific integration, our method introduces a standalone layer that spatially emphasizes high-importance regions in the input. We evaluated Attention Zoom on multiple CNN backbones using CIFAR-100 and TinyImageNet, showing consistent improvements in Top-1 and Top-5 cla…

This https://arxiv.org/abs/2506.05588 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csNE_…

Preprocessing Methods for Memristive Reservoir Computing for Image Recognition
Reservoir computing (RC) has attracted attention as an efficient recurrent neural network architecture due to its simplified training, requiring only its last perceptron readout layer to be trained. When implemented with memristors, RC systems benefit from their dynamic properties, which make them ideal for reservoir construction. However, achieving high performance in memristor-based RC remains challenging, as it critically depends on the input preprocessing method and reservoir size. Despite …

Understanding and Controlling Repetition Neurons and Induction Heads in In-Context Learning
Nhi Hoai Doan, Tatsuya Hiraoka, Kentaro Inui
https://arxiv.org/abs/2507.07810

Understanding and Controlling Repetition Neurons and Induction Heads in In-Context Learning
This paper investigates the relationship between large language models' (LLMs) ability to recognize repetitive input patterns and their performance on in-context learning (ICL). In contrast to prior work that has primarily focused on attention heads, we examine this relationship from the perspective of skill neurons, specifically repetition neurons. Our experiments reveal that the impact of these neurons on ICL performance varies depending on the depth of the layer in which they reside. By comp…

Generalization performance of narrow one-hidden layer networks in the teacher-student setting
Jean Barbier, Federica Gerace, Alessandro Ingrosso, Clarissa Lauditi, Enrico M. Malatesta, Gibbs Nwemadji, Rodrigo P\'erez Ortiz
https://arxiv.org/abs/2507.00629

Generalization performance of narrow one-hidden layer networks in the teacher-student setting
Understanding the generalization abilities of neural networks for simple input-output distributions is crucial to account for their learning performance on real datasets. The classical teacher-student setting, where a network is trained from data obtained thanks to a label-generating teacher model, serves as a perfect theoretical test bed. In this context, a complete theoretical account of the performance of fully connected one-hidden layer networks in the presence of generic activation functio…

PatchDSU: Uncertainty Modeling for Out of Distribution Generalization in Keyword Spotting
Bronya Roni Chernyak, Yael Segal, Yosi Shrem, Joseph Keshet
https://arxiv.org/abs/2508.03190

PatchDSU: Uncertainty Modeling for Out of Distribution Generalization in Keyword Spotting
Deep learning models excel at many tasks but rely on the assumption that training and test data follow the same distribution. This assumption often does not hold in real-world speech systems, where distribution shifts are common due to varying environments, recording conditions, and speaker diversity. The method of Domain Shifts with Uncertainty (DSU) augments the input of each neural network layer based on the input feature statistics. It addresses the problem of out-of-domain generalization…

This https://arxiv.org/abs/2503.14372 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Online ResNet-Based Adaptive Control for Nonlinear Target Tracking
A generalized ResNet architecture for adaptive control of nonlinear systems with black box uncertainties is developed. The approach overcomes limitations in existing methods by incorporating pre-activation shortcut connections and a zeroth layer block that accommodates different input-output dimensions. The developed Lyapunov-based adaptation law establishes exponential convergence to a neighborhood of the target state despite unknown dynamics and disturbances. Furthermore, the theoretical resu…

Trainable dynamical masking for readout-free optical computing
S. Bogdanov, E. Manuylovich, S. K. Turitsyn
https://arxiv.org/abs/2505.23464 https://…

Trainable dynamical masking for readout-free optical computing
Nonlinear systems, transforming an input signal into a high-dimensional output feature space, can be used for non-conventional computing. This approach, however, requires a change of system parameters during training rather than coefficients in a software program. We propose here to use available off-the-shelf high-speed optical communication devices and technologies to implement a trainable dynamical mask in addition to or even instead of the traditional readout layer for extreme learning mach…

Preprocessing Methods for Memristive Reservoir Computing for Image Recognition
Rishona Daniels, Duna Wattad, Ronny Ronen, David Saad, Shahar Kvatinsky
https://arxiv.org/abs/2506.05588

Preprocessing Methods for Memristive Reservoir Computing for Image Recognition
Reservoir computing (RC) has attracted attention as an efficient recurrent neural network architecture due to its simplified training, requiring only its last perceptron readout layer to be trained. When implemented with memristors, RC systems benefit from their dynamic properties, which make them ideal for reservoir construction. However, achieving high performance in memristor-based RC remains challenging, as it critically depends on the input preprocessing method and reservoir size. Despite …

Emergent Semantics Beyond Token Embeddings: Transformer LMs with Frozen Visual Unicode Representations
A. Bochkov
https://arxiv.org/abs/2507.04886 https://…

Emergent Semantics Beyond Token Embeddings: Transformer LMs with Frozen Visual Unicode Representations
Understanding the locus of semantic representation in large language models (LLMs) is crucial for interpretability and architectural innovation. The dominant paradigm posits that trainable input embeddings serve as foundational "meaning vectors." This paper challenges that view. We construct Transformer models where the embedding layer is entirely frozen, with vectors derived not from data, but from the visual structure of Unicode glyphs. These non-semantic, precomputed visual embeddings are fi…

Modeling the Path of Structural Strategic Deterrence: A Sand Table Simulation and Research Report on China's Military-Industrial Capability System against the United States Based on Rare Earth Supply Disconnection
Wei Meng
https://arxiv.org/abs/2505.21579

Modeling the Path of Structural Strategic Deterrence: A Sand Table Simulation and Research Report on China's Military-Industrial Capability System against the United States Based on Rare Earth Supply Disconnection
This study proposes a systematic non-kinetic deterrence path modeling framework based on strategic rare earth supply cut-off, aiming to assess the strategic effects of China's export control policy against the United States at the military system level. The model adopts a four-layer structure of "policy input -- resource node -- equipment system -- capability output" and integrates path dependency modeling, degradation function design, and capability lag prediction mechanisms to form a strategi…

Liquid and solid layers in a thermal deep learning machine
Gang Huang, Lai Shun Chan, Hajime Yoshino, Ge Zhang, Yuliang Jin
https://arxiv.org/abs/2506.06789

Liquid and solid layers in a thermal deep learning machine
Based on deep neural networks (DNNs), deep learning has been successfully applied to many problems, but its mechanism is still not well understood -- especially the reason why over-parametrized DNNs can generalize. A recent statistical mechanics theory on supervised learning by a prototypical multi-layer perceptron (MLP) on some artificial learning scenarios predicts that adjustable parameters of over-parametrized MLPs become strongly constrained by the training data close to the input/output b…

ICWLM: A Multi-Task Wireless Large Model via In-Context Learning
Yuxuan Wen, Xiaoming Chen, Maojun Zhang, Zhaoyang Zhang
https://arxiv.org/abs/2507.18167 https://

ICWLM: A Multi-Task Wireless Large Model via In-Context Learning
The rapid evolution of wireless communication technologies, particularly massive multiple-input multiple-output (mMIMO) and millimeter-wave (mmWave), introduces significant network complexity and computational demands. Significant research efforts have been made to improve physical layer performance by resorting to deep learning (DL) methods, which, however, are usually task-specific and struggle with data scarcity and generalization. To address these challenges, we propose a novel In-Context W…

Game-Theoretic Gradient Control for Robust Neural Network Training
Maria Zaitseva, Ivan Tomilov, Natalia Gusarova
https://arxiv.org/abs/2507.19143 https://…

Game-Theoretic Gradient Control for Robust Neural Network Training
Feed-forward neural networks (FFNNs) are vulnerable to input noise, reducing prediction performance. Existing regularization methods like dropout often alter network architecture or overlook neuron interactions. This study aims to enhance FFNN noise robustness by modifying backpropagation, interpreted as a multi-agent game, and exploring controlled target variable noising. Our "gradient dropout" selectively nullifies hidden layer neuron gradients with probability 1 - p during backpropagation, w…