Tootfinder

Backpropagation in unstable diffusions
Angxiu Ni
https://arxiv.org/abs/2507.21497 https://arxiv.org/pdf/2507.21497

Backpropagation in unstable diffusions
We derive the adjoint path-kernel method for the parameter-gradient of SDEs, where the observable is averaged at a particular time or over the stationary measure. Its cost is almost independent of the number of parameters; it extends the conventional backpropagation method to cases with gradient explosion. It works for non-hyperbolic systems with multiplicative noise controlled by parameters. We derive a Monte-Carlo-type algorithm and demonstrate it on the 40-dimensional Lorenz 96 system.

Supervised Stochastic Gradient Algorithms for Multi-Trial Source Separation
Ronak Mehta, Mateus Piovezan Otto, Noah Stanis, Azadeh Yazdan-Shahmorad, Zaid Harchaoui
https://arxiv.org/abs/2508.20618

Supervised Stochastic Gradient Algorithms for Multi-Trial Source Separation
We develop a stochastic algorithm for independent component analysis that incorporates multi-trial supervision, which is available in many scientific contexts. The method blends a proximal gradient-type algorithm in the space of invertible matrices with joint learning of a prediction model through backpropagation. We illustrate the proposed algorithm on synthetic and real data experiments. In particular, owing to the additional supervision, we observe an increased success rate of the non-convex…

This https://arxiv.org/abs/2412.06481 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

DeePC-Hunt: Data-enabled Predictive Control Hyperparameter Tuning via Differentiable Optimization
This paper introduces Data-enabled Predictive Control Hyperparameter Tuning via Differentiable Optimization (DeePC-Hunt), a backpropagation-based method for automatic hyperparameter tuning of the DeePC algorithm. The necessity for such a method arises from the importance of hyperparameter selection to achieve satisfactory closed-loop DeePC performance. The standard methods for hyperparameter selection are to either optimize the open-loop performance, or use manual guess-and-check. Optimizing th…

Trainable Joint Time-Vertex Fractional Fourier Transform
Ziqi Yan, Zhichao Zhang
https://arxiv.org/abs/2507.21527 https://arxiv.org/pdf/2507.21527

Trainable Joint Time-Vertex Fractional Fourier Transform
To address limitations of the graph fractional Fourier transform (GFRFT) Wiener filtering and the traditional joint time-vertex fractional Fourier transform (JFRFT) Wiener filtering, this study proposes a filtering method based on the hyper-differential form of the JFRFT. The gradient backpropagation mechanism is employed to enable the adaptive selection of transform order pair and filter coefficients. First, leveraging the hyper-differential form of the GFRFT and the fractional Fourier transfo…

Game-Theoretic Gradient Control for Robust Neural Network Training
Maria Zaitseva, Ivan Tomilov, Natalia Gusarova
https://arxiv.org/abs/2507.19143 https://…

Game-Theoretic Gradient Control for Robust Neural Network Training
Feed-forward neural networks (FFNNs) are vulnerable to input noise, reducing prediction performance. Existing regularization methods like dropout often alter network architecture or overlook neuron interactions. This study aims to enhance FFNN noise robustness by modifying backpropagation, interpreted as a multi-agent game, and exploring controlled target variable noising. Our "gradient dropout" selectively nullifies hidden layer neuron gradients with probability 1 - p during backpropagation, w…

Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignment
Youjia Zhang, Youngeun Kim, Young-Geun Choi, Hongyeob Kim, Huiling Liu, Sungeun Hong
https://arxiv.org/abs/2508.15568

Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignment
Test-time adaptation (TTA) enhances the zero-shot robustness under distribution shifts by leveraging unlabeled test data during inference. Despite notable advances, several challenges still limit its broader applicability. First, most methods rely on backpropagation or iterative optimization, which limits scalability and hinders real-time deployment. Second, they lack explicit modeling of class-conditional feature distributions. This modeling is crucial for producing reliable decision boundarie…

This https://arxiv.org/abs/2412.06481 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

DeePC-Hunt: Data-enabled Predictive Control Hyperparameter Tuning via Differentiable Optimization
This paper introduces Data-enabled Predictive Control Hyperparameter Tuning via Differentiable Optimization (DeePC-Hunt), a backpropagation-based method for automatic hyperparameter tuning of the DeePC algorithm. The necessity for such a method arises from the importance of hyperparameter selection to achieve satisfactory closed-loop DeePC performance. The standard methods for hyperparameter selection are to either optimize the open-loop performance, or use manual guess-and-check. Optimizing th…

Riemannian Optimization on Tree Tensor Networks with Application in Machine Learning
Marius Willner, Marco Trenti, Dirk Lebiedz
https://arxiv.org/abs/2507.21726 https://

Riemannian Optimization on Tree Tensor Networks with Application in Machine Learning
Tree tensor networks (TTNs) are widely used in low-rank approximation and quantum many-body simulation. In this work, we present a formal analysis of the differential geometry underlying TTNs. Building on this foundation, we develop efficient first- and second-order optimization algorithms that exploit the intrinsic quotient structure of TTNs. Additionally, we devise a backpropagation algorithm for training TTNs in a kernel learning setting. We validate our methods through numerical experiments…

Advanced For-Loop for QML algorithm search
FuTe Wong
https://arxiv.org/abs/2506.18260 https://arxiv.org/pdf/2506.18260

Advanced For-Loop for QML algorithm search
This paper introduces an advanced framework leveraging Large Language Model-based Multi-Agent Systems (LLMMA) for the automated search and optimization of Quantum Machine Learning (QML) algorithms. Inspired by Google DeepMind's FunSearch, the proposed system works on abstract level to iteratively generates and refines quantum transformations of classical machine learning algorithms (concepts), such as the Multi-Layer Perceptron, forward-forward and backpropagation algorithms. As a proof of conc…

Adversarial Disentanglement by Backpropagation with Physics-Informed Variational Autoencoder
Ioannis Christoforos Koune, Alice Cicirello
https://arxiv.org/abs/2506.13658

Adversarial Disentanglement by Backpropagation with Physics-Informed Variational Autoencoder
Inference and prediction under partial knowledge of a physical system is challenging, particularly when multiple confounding sources influence the measured response. Explicitly accounting for these influences in physics-based models is often infeasible due to epistemic uncertainty, cost, or time constraints, resulting in models that fail to accurately describe the behavior of the system. On the other hand, data-driven machine learning models such as variational autoencoders are not guaranteed t…

Integrated photonic deep neural network with end-to-end on-chip backpropagation training
Farshid Ashtiani, Mohamad Hossein Idjadi, Kwangwoong Kim
https://arxiv.org/abs/2506.14575 …

Integrated photonic deep neural network with end-to-end on-chip backpropagation training
Integrated photonic neural networks (PNNs) have demonstrated significant potential to complement the digital electronic counterparts [1-3]. Nevertheless, robust and repeatable performance of scalable integrated PNNs is directly tied to the quality of their training. Error backpropagation (BP), which relies on nonlinear activation gradient computation, is the mainstream algorithm to train digital neural networks due to its scalability, versatility, and implementation efficiency [4]. Consequently…

Replaced article(s) found for cs.LG. https://arxiv.org/list/cs.LG/new
[1/5]:
- Backpropagation Through Time For Networks With Long-Term Dependencies
George Bird, Maxim E. Polivoda

Timestep-Compressed Attack on Spiking Neural Networks through Timestep-Level Backpropagation
Donghwa Kang, Doohyun Kim, Sang-Ki Ko, Jinkyu Lee, Hyeongboo Baek, Brent ByungHoon Kang
https://arxiv.org/abs/2508.13812

Timestep-Compressed Attack on Spiking Neural Networks through Timestep-Level Backpropagation
State-of-the-art (SOTA) gradient-based adversarial attacks on spiking neural networks (SNNs), which largely rely on extending FGSM and PGD frameworks, face a critical limitation: substantial attack latency from multi-timestep processing, rendering them infeasible for practical real-time applications. This inefficiency stems from their design as direct extensions of ANN paradigms, which fail to exploit key SNN properties. In this paper, we propose the timestep-compressed attack (TCA), a novel fr…

When Pipelined In-Memory Accelerators Meet Spiking Direct Feedback Alignment: A Co-Design for Neuromorphic Edge Computing
Haoxiong Ren, Yangu He, Kwunhang Wong, Rui Bao, Ning Lin, Zhongrui Wang, Dashan Shang
https://arxiv.org/abs/2507.15603

When Pipelined In-Memory Accelerators Meet Spiking Direct Feedback Alignment: A Co-Design for Neuromorphic Edge Computing
Spiking Neural Networks (SNNs) are increasingly favored for deployment on resource-constrained edge devices due to their energy-efficient and event-driven processing capabilities. However, training SNNs remains challenging because of the computational intensity of traditional backpropagation algorithms adapted for spike-based systems. In this paper, we propose a novel software-hardware co-design that introduces a hardware-friendly training algorithm, Spiking Direct Feedback Alignment (SDFA) and…

Eig-PIELM: A Mesh-Free Approach for Efficient Eigen-Analysis with Physics-Informed Extreme Learning Machines
Rishi Mishra, Smriti, Ganapathy Krishnamurthi, Balaji Srinivasan, Sundararajan Natarajan
https://arxiv.org/abs/2508.15343

Eig-PIELM: A Mesh-Free Approach for Efficient Eigen-Analysis with Physics-Informed Extreme Learning Machines
In this work, a novel Eig-PIELM framework is proposed that extends physics-informed extreme learning machine for an efficient and accurate solution of linear eigenvalue problems. The method reformulates the governing differential equations into a compact algebraic system solvable in a single step. Boundary conditions are enforced exactly via an algebraic projection onto the boundary-admissible subspace, eliminating the computational overhead of penalty parameters, and backpropagation while pres…

Quantum-Inspired Differentiable Integral Neural Networks (QIDINNs): A Feynman-Based Architecture for Continuous Learning Over Streaming Data
Oscar Boullosa Dapena
https://arxiv.org/abs/2506.12111

Quantum-Inspired Differentiable Integral Neural Networks (QIDINNs): A Feynman-Based Architecture for Continuous Learning Over Streaming Data
Real-time continuous learning over streaming data remains a central challenge in deep learning and AI systems. Traditional gradient-based models such as backpropagation through time (BPTT) face computational and stability limitations when dealing with temporally unbounded data. In this paper, we introduce a novel architecture, Quantum-Inspired Differentiable Integral Neural Networks (QIDINNs), which leverages the Feynman technique of differentiation under the integral sign to formulate neural u…

Training nonlinear optical neural networks with Scattering Backpropagation
Nicola Dal Cin, Florian Marquardt, Clara C. Wanjura
https://arxiv.org/abs/2508.11750 https://

Training nonlinear optical neural networks with Scattering Backpropagation
As deep learning applications continue to deploy increasingly large artificial neural networks, the associated high energy demands are creating a need for alternative neuromorphic approaches. Optics and photonics are particularly compelling platforms as they offer high speeds and energy efficiency. Neuromorphic systems based on nonlinear optics promise high expressivity with a minimal number of parameters. However, so far, there is no efficient and generic physics-based training method allowing…

Crosslisted article(s) found for cond-mat.mes-hall. https://arxiv.org/list/cond-mat.mes-hall/new
[1/1]:
- Training nonlinear optical neural networks with Scattering Backpropagation
Nicola Dal Cin, Florian Marquardt, Clara C. Wanjura

Gradients of unitary optical neural networks using parameter-shift rule
Jinzhe Jiang, Yaqian Zhao, Xin Zhang, Chen Li, Yunlong Yu, Hailing Liu
https://arxiv.org/abs/2506.11565

Gradients of unitary optical neural networks using parameter-shift rule
This paper explores the application of the parameter-shift rule (PSR) for computing gradients in unitary optical neural networks (UONNs). While backpropagation has been fundamental to training conventional neural networks, its implementation in optical neural networks faces significant challenges due to the physical constraints of optical systems. We demonstrate how PSR, which calculates gradients by evaluating functions at shifted parameter values, can be effectively adapted for training UONNs…

Crosslisted article(s) found for cond-mat.dis-nn. https://arxiv.org/list/cond-mat.dis-nn/new
[1/1]:
- Training nonlinear optical neural networks with Scattering Backpropagation
Nicola Dal Cin, Florian Marquardt, Clara C. Wanjura

Bio-Inspired Artificial Neural Networks based on Predictive Coding
Davide Casnici, Charlotte Frenkel, Justin Dauwels
https://arxiv.org/abs/2508.08762 https://

Bio-Inspired Artificial Neural Networks based on Predictive Coding
Backpropagation (BP) of errors is the backbone training algorithm for artificial neural networks (ANNs). It updates network weights through gradient descent to minimize a loss function representing the mismatch between predictions and desired outputs. BP uses the chain rule to propagate the loss gradient backward through the network hierarchy, allowing efficient weight updates. However, this process requires weight updates at every layer to rely on a global error signal generated at the network…

A Self-Ensemble Inspired Approach for Effective Training of Binary-Weight Spiking Neural Networks
Qingyan Meng, Mingqing Xiao, Zhengyu Ma, Huihui Zhou, Yonghong Tian, Zhouchen Lin
https://arxiv.org/abs/2508.12609

A Self-Ensemble Inspired Approach for Effective Training of Binary-Weight Spiking Neural Networks
Spiking Neural Networks (SNNs) are a promising approach to low-power applications on neuromorphic hardware due to their energy efficiency. However, training SNNs is challenging because of the non-differentiable spike generation function. To address this issue, the commonly used approach is to adopt the backpropagation through time framework, while assigning the gradient of the non-differentiable function with some surrogates. Similarly, Binary Neural Networks (BNNs) also face the non-differenti…

A Scalable Hybrid Training Approach for Recurrent Spiking Neural Networks
Maximilian Baronig, Yeganeh Bahariasl, Ozan \"Ozdenizci, Robert Legenstein
https://arxiv.org/abs/2506.14464

A Scalable Hybrid Training Approach for Recurrent Spiking Neural Networks
Recurrent spiking neural networks (RSNNs) can be implemented very efficiently in neuromorphic systems. Nevertheless, training of these models with powerful gradient-based learning algorithms is mostly performed on standard digital hardware using Backpropagation through time (BPTT). However, BPTT has substantial limitations. It does not permit online training and its memory consumption scales linearly with the number of computation steps. In contrast, learning methods using forward propagation o…

Neural Jumps for Option Pricing
Duosi Zheng, Hanzhong Guo, Yanchu Liu, Wei Huang
https://arxiv.org/abs/2506.05137 https://arxiv.org/p…

Neural Jumps for Option Pricing
Recognizing the importance of jump risk in option pricing, we propose a neural jump stochastic differential equation model in this paper, which integrates neural networks as parameter estimators in the conventional jump diffusion model. To overcome the problem that the backpropagation algorithm is not compatible with the jump process, we use the Gumbel-Softmax method to make the jump parameter gradient learnable. We examine the proposed model using both simulated data and S&P 500 index options.…

Gradient Similarity Surgery in Multi-Task Deep Learning
Thomas Borsani, Andrea Rosani, Giuseppe Nicosia, Giuseppe Di Fatta
https://arxiv.org/abs/2506.06130

Gradient Similarity Surgery in Multi-Task Deep Learning
The multi-task learning ($MTL$) paradigm aims to simultaneously learn multiple tasks within a single model capturing higher-level, more general hidden patterns that are shared by the tasks. In deep learning, a significant challenge in the backpropagation training process is the design of advanced optimisers to improve the convergence speed and stability of the gradient descent learning rule. In particular, in multi-task deep learning ($MTDL$) the multitude of tasks may generate potentially conf…

This https://arxiv.org/abs/2505.00533 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

Test-time Correlation Alignment
Deep neural networks often degrade under distribution shifts. Although domain adaptation offers a solution, privacy constraints often prevent access to source data, making Test-Time Adaptation (TTA, which adapts using only unlabeled test data) increasingly attractive. However, current TTA methods still face practical challenges: (1) a primary focus on instance-wise alignment, overlooking CORrelation ALignment (CORAL) due to missing source correlations; (2) complex backpropagation operations for…

Tangma: A Tanh-Guided Activation Function with Learnable Parameters
Shreel Golwala
https://arxiv.org/abs/2507.10560 https://arxiv.org…

Tangma: A Tanh-Guided Activation Function with Learnable Parameters
Activation functions are key to effective backpropagation and expressiveness in deep neural networks. This work introduces Tangma, a new activation function that combines the smooth shape of the hyperbolic tangent with two learnable parameters: $α$, which shifts the curve's inflection point to adjust neuron activation, and $γ$, which adds linearity to preserve weak gradients and improve training stability. Tangma was evaluated on MNIST and CIFAR-10 using custom networks composed of convolutio…

Chameleon: A MatMul-Free Temporal Convolutional Network Accelerator for End-to-End Few-Shot and Continual Learning from Sequential Data
Douwe den Blanken, Charlotte Frenkel
https://arxiv.org/abs/2505.24852

Chameleon: A MatMul-Free Temporal Convolutional Network Accelerator for End-to-End Few-Shot and Continual Learning from Sequential Data
On-device learning at the edge enables low-latency, private personalization with improved long-term robustness and reduced maintenance costs. Yet, achieving scalable, low-power end-to-end on-chip learning, especially from real-world sequential data with a limited number of examples, is an open challenge. Indeed, accelerators supporting error backpropagation optimize for learning performance at the expense of inference efficiency, while simplified learning algorithms often fail to reach acceptab…

This https://arxiv.org/abs/2501.09976 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csNE_…

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
Backpropagation is the foundational algorithm for training neural networks and a key driver of deep learning's success. However, its biological plausibility has been challenged due to three primary limitations: weight symmetry, reliance on global error signals, and the dual-phase nature of training, as highlighted by the existing literature. Although various alternative learning approaches have been proposed to address these issues, most either fail to satisfy all three criteria simultaneously …

Spatio-Temporal Decoupled Learning for Spiking Neural Networks
Chenxiang Ma, Xinyi Chen, Kay Chen Tan, Jibin Wu
https://arxiv.org/abs/2506.01117 https://…

Spatio-Temporal Decoupled Learning for Spiking Neural Networks
Spiking neural networks (SNNs) have gained significant attention for their potential to enable energy-efficient artificial intelligence. However, effective and efficient training of SNNs remains an unresolved challenge. While backpropagation through time (BPTT) achieves high accuracy, it incurs substantial memory overhead. In contrast, biologically plausible local learning methods are more memory-efficient but struggle to match the accuracy of BPTT. To bridge this gap, we propose spatio-tempora…