Tootfinder

ReBaNO: Reduced Basis Neural Operator Mitigating Generalization Gaps and Achieving Discretization Invariance
Haolan Zheng, Yanlai Chen, Jiequn Han, Yue Yu
https://arxiv.org/abs/2509.09611

ReBaNO: Reduced Basis Neural Operator Mitigating Generalization Gaps and Achieving Discretization Invariance
We propose a novel data-lean operator learning algorithm, the Reduced Basis Neural Operator (ReBaNO), to solve a group of PDEs with multiple distinct inputs. Inspired by the Reduced Basis Method and the recently introduced Generative Pre-Trained Physics-Informed Neural Networks, ReBaNO relies on a mathematically rigorous greedy algorithm to build its network structure offline adaptively from the ground up. Knowledge distillation via task-specific activation function allows ReBaNO to have a comp…

Nuclear Mass Predictions Using a Neural Network with Additive Gaussian Process Regression-Optimized Activation Functions
H. X. Liu, S. Manzhos, X. H. Wu
https://arxiv.org/abs/2509.08314

Nuclear Mass Predictions Using a Neural Network with Additive Gaussian Process Regression-Optimized Activation Functions
Nuclear masses are machine-learned as a function of proton and neutron numbers. The neural network with additive Gaussian process regression-optimized activation functions (GPR-NN) method is employed for the first time for this purpose. GPR-NN combines the advantages of both neural networks and Gaussian process regression, in that it possesses the expressive power of an NN, in principle allowing modeling any kind of dependence of nuclear mass on the features, and robustness of a linear regr…

20-GHz bandwidth optical activation function based on a semiconductor laser
Hai-Fei Guo, Zheng-Can Sun, Yi-Wei Shen, Rui-Qian Li, Xing Li, Cheng Wang
https://arxiv.org/abs/2507.00468

20-GHz bandwidth optical activation function based on a semiconductor laser
Optical neural networks usually execute the linear multiply-accumulate operation in the optical domain, whereas the nonlinear activation function is mostly implemented in the digital or electrical domain. Here we demonstrate a broadband activation function operated in the optical domain. The optical activation function (OAF) is achieved by a semiconductor laser with optical injection, which is operated in the unstable-locked regime above the Hopf bifurcation. The operation bandwidth of the OAF …

Learning quadratic neural networks in high dimensions: SGD dynamics and scaling laws
G\'erard Ben Arous, Murat A. Erdogdu, N. Mert Vural, Denny Wu
https://arxiv.org/abs/2508.03688

Learning quadratic neural networks in high dimensions: SGD dynamics and scaling laws
We study the optimization and sample complexity of gradient-based training of a two-layer neural network with quadratic activation function in the high-dimensional regime, where the data is generated as $y \propto \sum_{j=1}^{r}λ_j σ\left(\langle \boldsymbol{θ_j}, \boldsymbol{x}\rangle\right), \boldsymbol{x} \sim N(0,\boldsymbol{I}_d)$, $σ$ is the 2nd Hermite polynomial, and $\lbrace\boldsymbolθ_j \rbrace_{j=1}^{r} \subset \mathbb{R}^d$ are orthonormal signal directions. We consider the ex…

Novel Complex-Valued Hopfield Neural Networks with Phase and Magnitude Quantization
Garimella Ramamurthy, Marcos Eduardo Valle, Tata Jagannadha Swamy
https://arxiv.org/abs/2507.00461

Novel Complex-Valued Hopfield Neural Networks with Phase and Magnitude Quantization
This research paper introduces two novel complex-valued Hopfield neural networks (CvHNNs) that incorporate phase and magnitude quantization. The first CvHNN employs a ceiling-type activation function that operates on the rectangular coordinate representation of the complex net contribution. The second CvHNN similarly incorporates phase and magnitude quantization but utilizes a ceiling-type activation function based on the polar coordinate representation of the complex net contribution. The prop…

I guess it's always nice when someone cites a paper. But too often, it's very clear that they haven't read it. If they had done, they would have realised that they were writing stuff that that I showed to be wrong in 1987. Sigh.
The extent of innumeracy in pharmacology is worrying.

Frontiers | The flexibility of SABRE, a new quantitative receptor function model, when fitting challenging concentration-effect data
The Signal Amplification, Binding affinity, and Receptor-activation Efficacy (SABRE) model is the most recent general and quantitative model of receptor func...

Beyond ReLU: How Activations Affect Neural Kernels and Random Wide Networks
David Holzm\"uller, Max Sch\"olpple
https://arxiv.org/abs/2506.22429 …

Beyond ReLU: How Activations Affect Neural Kernels and Random Wide Networks
While the theory of deep learning has made some progress in recent years, much of it is limited to the ReLU activation function. In particular, while the neural tangent kernel (NTK) and neural network Gaussian process kernel (NNGP) have given theoreticians tractable limiting cases of fully connected neural networks, their properties for most activation functions except for powers of the ReLU function are poorly understood. Our main contribution is to provide a more general characterization of t…

Adiabatic Capacitive Neuron: An Energy-Efficient Functional Unit for Artificial Neural Networks
Sachin Maheshwari, Mike Smart, Himadri Singh Raghav, Themis Prodromakis, Alexander Serb
https://arxiv.org/abs/2507.00831

Adiabatic Capacitive Neuron: An Energy-Efficient Functional Unit for Artificial Neural Networks
This paper introduces a new, highly energy-efficient, Adiabatic Capacitive Neuron (ACN) hardware implementation of an Artificial Neuron (AN) with improved functionality, accuracy, robustness and scalability over previous work. The paper describes the implementation of a \mbox{12-bit} single neuron, with positive and negative weight support, in an $\mathbf{0.18μm}$ CMOS technology. The paper also presents a new Threshold Logic (TL) design for a binary AN activation function that generates a low…

Tangma: A Tanh-Guided Activation Function with Learnable Parameters
Shreel Golwala
https://arxiv.org/abs/2507.10560 https://arxiv.org…

Tangma: A Tanh-Guided Activation Function with Learnable Parameters
Activation functions are key to effective backpropagation and expressiveness in deep neural networks. This work introduces Tangma, a new activation function that combines the smooth shape of the hyperbolic tangent with two learnable parameters: $α$, which shifts the curve's inflection point to adjust neuron activation, and $γ$, which adds linearity to preserve weak gradients and improve training stability. Tangma was evaluated on MNIST and CIFAR-10 using custom networks composed of convolutio…

Activated Backstepping with Control Barrier Functions for the Safe Navigation of Automated Vehicles
Laszlo Gacsi, Max H. Cohen, Tamas G. Molnar
https://arxiv.org/abs/2508.20822 …

Activated Backstepping with Control Barrier Functions for the Safe Navigation of Automated Vehicles
This paper introduces a novel safety-critical control method through the synthesis of control barrier functions (CBFs) for systems with high-relative-degree safety constraints. By extending the procedure of CBF backstepping, we propose activated backstepping - a constructive method to synthesize valid CBFs. The novelty of our method is the incorporation of an activation function into the CBF, which offers less conservative safe sets in the state space than standard CBF backstepping. We demonstr…

Genericity of Polyak-Lojasiewicz Inequalities for Entropic Mean-Field Neural ODEs
Samuel Daudin, Fran\c{c}ois Delarue
https://arxiv.org/abs/2507.08486 http…

Genericity of Polyak-Lojasiewicz Inequalities for Entropic Mean-Field Neural ODEs
We address the behavior of idealized deep residual neural networks (ResNets), modeled via an optimal control problem set over continuity (or adjoint transport) equations. The continuity equations describe the statistical evolution of the features in the asymptotic regime where the layers of the network form a continuum. The velocity field is expressed through the network activation function, which is itself viewed as a function of the statistical distribution of the network parameters (weights …

Molecular Tools for Non-Planar Surface Chemistry
Taleana Huff, Brandon Blue, Terry McCallum, Mathieu Morin, Damian G. Allis, Rafik Addou, Jeremy Barton, Adam Bottomley, Doreen Cheng, Nina M. \'Culum, Michael Drew, Tyler Enright, Alan T. K. Godfrey, Ryan Groome, Aru J. Hill, Alex Inayeh, Matthew R. Kennedy, Robert J. Kirby, Mykhaylo Krykunov, Sam Lilak, Hadiya Ma, Cameron J. Mackie, Oliver MacLean, Jonathan Myall, Ryan Plumadore, Adam Powell, Henry Rodriguez, Luis Sandoval, Marc Sav…

Molecular Tools for Non-Planar Surface Chemistry
Scanning probe microscopy (SPM) investigations of on-surface chemistry on passivated silicon have only shown in-plane chemical reactions, and studies on bare silicon are limited in facilitating additional reactions post-molecular-attachment. Here, we enable subsequent reactions on Si(100) through selectively adsorbing 3D, silicon-specific "molecular tools". Following an activation step, the molecules present an out-of-plane radical that can function both to donate or accept molecular fragments,…

Solving Approximation Tasks with Greedy Deep Kernel Methods
Marian Klink, Tobias Ehring, Robin Herkert, Robin Lautenschlager, Dominik G\"oddeke, Bernard Haasdonk
https://arxiv.org/abs/2508.08759

Solving Approximation Tasks with Greedy Deep Kernel Methods
Kernel methods are versatile tools for function approximation and surrogate modeling. In particular, greedy techniques offer computational efficiency and reliability through inherent sparsity and provable convergence. Inspired by the success of deep neural networks and structured deep kernel networks, we consider deep, multilayer kernels for greedy approximation. This multilayer structure, consisting of linear kernel layers and optimizable kernel activation function layers in an alternating fas…

APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation
Ravin Kumar
https://arxiv.org/abs/2507.14270 https://

APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation
We propose the APTx Neuron, a novel, unified neural computation unit that integrates non-linear activation and linear transformation into a single trainable expression. The APTx Neuron is derived from the APTx activation function, thereby eliminating the need for separate activation layers and making the architecture both computationally efficient and elegant. The proposed neuron follows the functional form $y = \sum_{i=1}^{n} ((α_i + \tanh(β_i x_i)) \cdot γ_i x_i) + δ$, where all parameter…

Emergence of Quantised Representations Isolated to Anisotropic Functions
George Bird
https://arxiv.org/abs/2507.12070 https://arxiv.o…

Emergence of Quantised Representations Isolated to Anisotropic Functions
This paper describes a novel methodology for determining representational alignment, developed upon the existing Spotlight Resonance method. Using this, it is found that algebraic symmetries of network primitives are a strong predictor for task-agnostic structure in representations. Particularly, this new tool is used to gain insight into how discrete representations can form and arrange in autoencoder models, through an ablation study where only the activation function is altered. Representati…

On the role of non-linear latent features in bipartite generative neural networks
Tony Bonnaire, Giovanni Catania, Aur\'elien Decelle, Beatriz Seoane
https://arxiv.org/abs/2506.10552

On the role of non-linear latent features in bipartite generative neural networks
We investigate the phase diagram and memory retrieval capabilities of bipartite energy-based neural networks, namely Restricted Boltzmann Machines (RBMs), as a function of the prior distribution imposed on their hidden units - including binary, multi-state, and ReLU-like activations. Drawing connections to the Hopfield model and employing analytical tools from statistical physics of disordered systems, we explore how the architectural choices and activation functions shape the thermodynamic pro…

Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weights
Krishnakumar Balasubramanian, Nathan Ross
https://arxiv.org/abs/2507.12686

Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weights
We study the Finite-Dimensional Distributions (FDDs) of deep neural networks with randomly initialized weights that have finite-order moments. Specifically, we establish Gaussian approximation bounds in the Wasserstein-$1$ norm between the FDDs and their Gaussian limit assuming a Lipschitz activation function and allowing the layer widths to grow to infinity at arbitrary relative rates. In the special case where all widths are proportional to a common scale parameter $n$ and there are $L-1$ hid…