Tootfinder

Activation Function Design Sustains Plasticity in Continual Learning
Lute Lillo, Nick Cheney
https://arxiv.org/abs/2509.22562 https://arxiv.org/pdf/2509.22…

Activation Function Design Sustains Plasticity in Continual Learning
In independent, identically distributed (i.i.d.) training regimes, activation functions have been benchmarked extensively, and their differences often shrink once model size and optimization are tuned. In continual learning, however, the picture is different: beyond catastrophic forgetting, models can progressively lose the ability to adapt (referred to as loss of plasticity) and the role of the non-linearity in this failure mode remains underexplored. We show that activation choice is a primar…

Maxout Polytopes
Andrei Balakin, Shelby Cox, Georg Loho, Bernd Sturmfels
https://arxiv.org/abs/2509.21286 https://arxiv.org/pdf/2509.21286

Maxout Polytopes
Maxout polytopes are defined by feedforward neural networks with maxout activation function and non-negative weights after the first layer. We characterize the parameter spaces and extremal f-vectors of maxout polytopes for shallow networks, and we study the separating hypersurfaces which arise when a layer is added to the network. We also show that maxout polytopes are cubical for generic networks without bottlenecks.

Dimension-Free Minimax Rates for Learning Pairwise Interactions in Attention-Style Models
Shai Zucker, Xiong Wang, Fei Lu, Inbar Seroussi
https://arxiv.org/abs/2510.11789 https:…

Dimension-Free Minimax Rates for Learning Pairwise Interactions in Attention-Style Models
We study the convergence rate of learning pairwise interactions in single-layer attention-style models, where tokens interact through a weight matrix and a non-linear activation function. We prove that the minimax rate is $M^{-\frac{2β}{2β+1}}$ with $M$ being the sample size, depending only on the smoothness $β$ of the activation, and crucially independent of token count, ambient dimension, or rank of the weight matrix. These results highlight a fundamental dimension-free statistical efficie…

Algebraic geometry of rational neural networks
Alexandros Grosdos, Elina Robeva, Maksym Zubkov
https://arxiv.org/abs/2509.11088 https://arxiv.org/pdf/2509.…

Algebraic geometry of rational neural networks
We study the expressivity of rational neural networks (RationalNets) through the lens of algebraic geometry. We consider rational functions that arise from a given RationalNet to be tuples of fractions of homogeneous polynomials of fixed degrees. For a given architecture, the neuromanifold is the set of all such expressible tuples. For RationalNets with one hidden layer and fixed activation function $1/x$, we characterize the dimension of the neuromanifold and provide defining equations for som…

Approximation by multivariate neural network operators in mixed norm space: theory and application
Priyanka Majethiya, Shivam Bajpeyi
https://arxiv.org/abs/2509.18740 https://…

Based on Deep Neural Networks: A Machine Learning-Assisted Channel Estimation Method for MIMO Systems
Haoran He
https://arxiv.org/abs/2510.11891 https://ar…

Based on Deep Neural Networks: A Machine Learning-Assisted Channel Estimation Method for MIMO Systems
This paper proposes a machine learning-assisted channel estimation approach for massive MIMO systems, leveraging DNNs to outperform traditional LS and MMSE methods. In 5G and beyond, accurate channel estimation mitigates pilot contamination and high mobility issues that harm system reliability. The proposed DNN architecture includes multi-layer perceptrons with ReLU activation, 3 hidden layers (256, 128, 64 neurons respectively), uses Adam optimizer (learning rate 1e-4) and MSE loss function. I…

Neural network-based singularity detection and applications
Nadiia Derevianko, Ioannis G. Kevrekidis, Felix Dietrich
https://arxiv.org/abs/2509.10110 https://

Neural network-based singularity detection and applications
We present a method for constructing a special type of shallow neural network that learns univariate meromorphic functions with pole-type singularities. Our method is based on using a finite set of Laurent coefficients as input information, which we compute by FFT, employing values of the investigated function on some contour $Γ$ in the complex plane. The primary components of our methodology are the following: (1) the adaptive construction of rational polynomial activation functions, (2) a no…