Tootfinder

APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation
Ravin Kumar
https://arxiv.org/abs/2507.14270 https://

APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation
We propose the APTx Neuron, a novel, unified neural computation unit that integrates non-linear activation and linear transformation into a single trainable expression. The APTx Neuron is derived from the APTx activation function, thereby eliminating the need for separate activation layers and making the architecture both computationally efficient and elegant. The proposed neuron follows the functional form $y = \sum_{i=1}^{n} ((α_i + \tanh(β_i x_i)) \cdot γ_i x_i) + δ$, where all parameter…

Tangma: A Tanh-Guided Activation Function with Learnable Parameters
Shreel Golwala
https://arxiv.org/abs/2507.10560 https://arxiv.org…

Tangma: A Tanh-Guided Activation Function with Learnable Parameters
Activation functions are key to effective backpropagation and expressiveness in deep neural networks. This work introduces Tangma, a new activation function that combines the smooth shape of the hyperbolic tangent with two learnable parameters: $α$, which shifts the curve's inflection point to adjust neuron activation, and $γ$, which adds linearity to preserve weak gradients and improve training stability. Tangma was evaluated on MNIST and CIFAR-10 using custom networks composed of convolutio…

20-GHz bandwidth optical activation function based on a semiconductor laser
Hai-Fei Guo, Zheng-Can Sun, Yi-Wei Shen, Rui-Qian Li, Xing Li, Cheng Wang
https://arxiv.org/abs/2507.00468

20-GHz bandwidth optical activation function based on a semiconductor laser
Optical neural networks usually execute the linear multiply-accumulate operation in the optical domain, whereas the nonlinear activation function is mostly implemented in the digital or electrical domain. Here we demonstrate a broadband activation function operated in the optical domain. The optical activation function (OAF) is achieved by a semiconductor laser with optical injection, which is operated in the unstable-locked regime above the Hopf bifurcation. The operation bandwidth of the OAF …

Emergence of Quantised Representations Isolated to Anisotropic Functions
George Bird
https://arxiv.org/abs/2507.12070 https://arxiv.o…

Emergence of Quantised Representations Isolated to Anisotropic Functions
This paper describes a novel methodology for determining representational alignment, developed upon the existing Spotlight Resonance method. Using this, it is found that algebraic symmetries of network primitives are a strong predictor for task-agnostic structure in representations. Particularly, this new tool is used to gain insight into how discrete representations can form and arrange in autoencoder models, through an ablation study where only the activation function is altered. Representati…

Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weights
Krishnakumar Balasubramanian, Nathan Ross
https://arxiv.org/abs/2507.12686

Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weights
We study the Finite-Dimensional Distributions (FDDs) of deep neural networks with randomly initialized weights that have finite-order moments. Specifically, we establish Gaussian approximation bounds in the Wasserstein-$1$ norm between the FDDs and their Gaussian limit assuming a Lipschitz activation function and allowing the layer widths to grow to infinity at arbitrary relative rates. In the special case where all widths are proportional to a common scale parameter $n$ and there are $L-1$ hid…

Genericity of Polyak-Lojasiewicz Inequalities for Entropic Mean-Field Neural ODEs
Samuel Daudin, Fran\c{c}ois Delarue
https://arxiv.org/abs/2507.08486 http…

Genericity of Polyak-Lojasiewicz Inequalities for Entropic Mean-Field Neural ODEs
We address the behavior of idealized deep residual neural networks (ResNets), modeled via an optimal control problem set over continuity (or adjoint transport) equations. The continuity equations describe the statistical evolution of the features in the asymptotic regime where the layers of the network form a continuum. The velocity field is expressed through the network activation function, which is itself viewed as a function of the statistical distribution of the network parameters (weights …

I guess it's always nice when someone cites a paper. But too often, it's very clear that they haven't read it. If they had done, they would have realised that they were writing stuff that that I showed to be wrong in 1987. Sigh.
The extent of innumeracy in pharmacology is worrying.

Frontiers | The flexibility of SABRE, a new quantitative receptor function model, when fitting challenging concentration-effect data
The Signal Amplification, Binding affinity, and Receptor-activation Efficacy (SABRE) model is the most recent general and quantitative model of receptor func...

BemaGANv2: A Tutorial and Comparative Survey of GAN-based Vocoders for Long-Term Audio Generation
Taesoo Park, Mungwi Jeong, Mingyu Park, Narae Kim, Junyoung Kim, Mujung Kim, Jisang Yoo, Hoyun Lee, Sanghoon Kim, Soonchul Kwon
https://arxiv.org/abs/2506.09487

BemaGANv2: A Tutorial and Comparative Survey of GAN-based Vocoders for Long-Term Audio Generation
This paper presents a tutorial-style survey and implementation guide of BemaGANv2, an advanced GAN-based vocoder designed for high-fidelity and long-term audio generation. Built upon the original BemaGAN architecture, BemaGANv2 incorporates major architectural innovations by replacing traditional ResBlocks in the generator with the Anti-aliased Multi-Periodicity composition (AMP) module, which internally applies the Snake activation function to better model periodic structures. In the discrimin…

Beyond ReLU: How Activations Affect Neural Kernels and Random Wide Networks
David Holzm\"uller, Max Sch\"olpple
https://arxiv.org/abs/2506.22429 …

Beyond ReLU: How Activations Affect Neural Kernels and Random Wide Networks
While the theory of deep learning has made some progress in recent years, much of it is limited to the ReLU activation function. In particular, while the neural tangent kernel (NTK) and neural network Gaussian process kernel (NNGP) have given theoreticians tractable limiting cases of fully connected neural networks, their properties for most activation functions except for powers of the ReLU function are poorly understood. Our main contribution is to provide a more general characterization of t…

Adiabatic Capacitive Neuron: An Energy-Efficient Functional Unit for Artificial Neural Networks
Sachin Maheshwari, Mike Smart, Himadri Singh Raghav, Themis Prodromakis, Alexander Serb
https://arxiv.org/abs/2507.00831

Adiabatic Capacitive Neuron: An Energy-Efficient Functional Unit for Artificial Neural Networks
This paper introduces a new, highly energy-efficient, Adiabatic Capacitive Neuron (ACN) hardware implementation of an Artificial Neuron (AN) with improved functionality, accuracy, robustness and scalability over previous work. The paper describes the implementation of a \mbox{12-bit} single neuron, with positive and negative weight support, in an $\mathbf{0.18μm}$ CMOS technology. The paper also presents a new Threshold Logic (TL) design for a binary AN activation function that generates a low…

On the role of non-linear latent features in bipartite generative neural networks
Tony Bonnaire, Giovanni Catania, Aur\'elien Decelle, Beatriz Seoane
https://arxiv.org/abs/2506.10552

On the role of non-linear latent features in bipartite generative neural networks
We investigate the phase diagram and memory retrieval capabilities of bipartite energy-based neural networks, namely Restricted Boltzmann Machines (RBMs), as a function of the prior distribution imposed on their hidden units - including binary, multi-state, and ReLU-like activations. Drawing connections to the Hopfield model and employing analytical tools from statistical physics of disordered systems, we explore how the architectural choices and activation functions shape the thermodynamic pro…

This https://arxiv.org/abs/2505.02580 has been replaced.
initial toot: https://mastoxiv.page/@ar…

Temperature and pressure reconstruction in turbulent Rayleigh-Bénard convection by Lagrangian velocities using PINN
Velocity, pressure, and temperature are the key variables for understanding thermal convection, and measuring them all is a complex task. In this paper, we demonstrate a method to reconstruct temperature and pressure fields based on given Lagrangian velocity data. A physics-informed neural network (PINN) based on a multilayer perceptron architecture and a periodic sine activation function is used to reconstruct both the temperature and the pressure for two cases of turbulent Rayleigh-Bénard co…

Immunometabolism at the Crossroads of Infection: Mechanistic and Systems-Level Perspectives from Host and Pathogen
Sunayana Malla, Nabia Shahreen, Rajib Saha
https://arxiv.org/abs/2506.02236

Immunometabolism at the Crossroads of Infection: Mechanistic and Systems-Level Perspectives from Host and Pathogen
The emerging field of immunometabolism has underscored the central role of metabolic pathways in orchestrating immune cell function. Far from being passive background processes, metabolic activities actively regulate key immune responses. Fundamental pathways such as glycolysis, the tricarboxylic acid (TCA) cycle, and oxidative phosphorylation critically shape the behavior of immune cells, influencing macrophage polarization, T cell activation, and dendritic cell function. In this review, we sy…

Novel Complex-Valued Hopfield Neural Networks with Phase and Magnitude Quantization
Garimella Ramamurthy, Marcos Eduardo Valle, Tata Jagannadha Swamy
https://arxiv.org/abs/2507.00461

Novel Complex-Valued Hopfield Neural Networks with Phase and Magnitude Quantization
This research paper introduces two novel complex-valued Hopfield neural networks (CvHNNs) that incorporate phase and magnitude quantization. The first CvHNN employs a ceiling-type activation function that operates on the rectangular coordinate representation of the complex net contribution. The second CvHNN similarly incorporates phase and magnitude quantization but utilizes a ceiling-type activation function based on the polar coordinate representation of the complex net contribution. The prop…

This https://arxiv.org/abs/2506.05197 has been replaced.
initial toot: https://mastoxiv.page/@arX…

Overlap Gap and Computational Thresholds in the Square Wave Perceptron
Square Wave Perceptrons (SWPs) form a class of neural network models with oscillating activation function that exhibit intriguing ``hardness'' properties in the high-dimensional limit at a fixed signal-to-noise ratio $α= O(1)$. In this work, we examine two key aspects of these models. The first is related to the so-called overlap-gap property, that is a disconnectivity feature of the geometry of the solution space of combinatorial optimization problems proven to cause the failure of a large fa…

Overlap Gap and Computational Thresholds in the Square Wave Perceptron
Marco Benedetti, Andrej Bogdanov, Enrico M. Malatesta, Marc M\'ezard, Gianmarco Perrupato, Alon Rosen, Nikolaj I. Schwartzbach, Riccardo Zecchina
https://arxiv.org/abs/2506.05197

Overlap Gap and Computational Thresholds in the Square Wave Perceptron
Square Wave Perceptrons (SWPs) form a class of neural network models with oscillating activation function that exhibit intriguing ``hardness'' properties in the high-dimensional limit at a fixed signal-to-noise ratio $α= O(1)$. In this work, we examine two key aspects of these models. The first is related to the so-called overlap-gap property, that is a disconnectivity feature of the geometry of the solution space of combinatorial optimization problems proven to cause the failure of a large fa…