Tootfinder

DIVEBATCH: Accelerating Model Training Through Gradient-Diversity Aware Batch Size Adaptation
Yuen Chen, Yian Wang, Hari Sundaram
https://arxiv.org/abs/2509.16173 https://

DIVEBATCH: Accelerating Model Training Through Gradient-Diversity Aware Batch Size Adaptation
The goal of this paper is to accelerate the training of machine learning models, a critical challenge since the training of large-scale deep neural models can be computationally expensive. Stochastic gradient descent (SGD) and its variants are widely used to train deep neural networks. In contrast to traditional approaches that focus on tuning the learning rate, we propose a novel adaptive batch size SGD algorithm, DiveBatch, that dynamically adjusts the batch size. Adapting the batch size is c…

Targeted Fine-Tuning of DNN-Based Receivers via Influence Functions
Marko Tuononen, Heikki Penttinen, Ville Hautam\"aki
https://arxiv.org/abs/2509.15950 https://

Targeted Fine-Tuning of DNN-Based Receivers via Influence Functions
We present the first use of influence functions for deep learning-based wireless receivers. Applied to DeepRx, a fully convolutional receiver, influence analysis reveals which training samples drive bit predictions, enabling targeted fine-tuning of poorly performing cases. We show that loss-relative influence with capacity-like binary cross-entropy loss and first-order updates on beneficial samples most consistently improves bit error rate toward genie-aided performance, outperforming random fi…

State-of-the-Art Dysarthric Speech Recognition with MetaICL for on-the-fly Personalization
Dhruuv Agarwal, Harry Zhang, Yang Yu, Quan Wang
https://arxiv.org/abs/2509.15516 https…

State-of-the-Art Dysarthric Speech Recognition with MetaICL for on-the-fly Personalization
Personalizing Automatic Speech Recognition (ASR) for dysarthric speech is crucial but challenging due to training and storing of individual user adapters. We propose a hybrid meta-training method for a single model, excelling in zero-shot and few-shot on-the-fly personalization via in-context learning (ICL). Measuring Word Error Rate (WER) on state-of-the-art subsets, the model achieves 13.9% WER on Euphonia which surpasses speaker-independent baselines (17.5% WER) and rivals user-specific pers…

Dimension-Free Minimax Rates for Learning Pairwise Interactions in Attention-Style Models
Shai Zucker, Xiong Wang, Fei Lu, Inbar Seroussi
https://arxiv.org/abs/2510.11789 https:…

Dimension-Free Minimax Rates for Learning Pairwise Interactions in Attention-Style Models
We study the convergence rate of learning pairwise interactions in single-layer attention-style models, where tokens interact through a weight matrix and a non-linear activation function. We prove that the minimax rate is $M^{-\frac{2β}{2β+1}}$ with $M$ being the sample size, depending only on the smoothness $β$ of the activation, and crucially independent of token count, ambient dimension, or rank of the weight matrix. These results highlight a fundamental dimension-free statistical efficie…

Im still "e-learning" how to generate aura and hype moments at a market-competitive rate so bear with me

Operand Quant: A Single-Agent Architecture for Autonomous Machine Learning Engineering
Arjun Sahney, Ram Gorthi, Cezary {\L}astowski, Javier Vega
https://arxiv.org/abs/2510.11694

Operand Quant: A Single-Agent Architecture for Autonomous Machine Learning Engineering
We present Operand Quant, a single-agent, IDE-based architecture for autonomous machine learning engineering (MLE). Operand Quant departs from conventional multi-agent orchestration frameworks by consolidating all MLE lifecycle stages -- exploration, modeling, experimentation, and deployment -- within a single, context-aware agent. On the MLE-Benchmark (2025), Operand Quant achieved a new state-of-the-art (SOTA) result, with an overall medal rate of 0.3956 +/- 0.0565 across 75 problems -- the h…

Based on Deep Neural Networks: A Machine Learning-Assisted Channel Estimation Method for MIMO Systems
Haoran He
https://arxiv.org/abs/2510.11891 https://ar…

Based on Deep Neural Networks: A Machine Learning-Assisted Channel Estimation Method for MIMO Systems
This paper proposes a machine learning-assisted channel estimation approach for massive MIMO systems, leveraging DNNs to outperform traditional LS and MMSE methods. In 5G and beyond, accurate channel estimation mitigates pilot contamination and high mobility issues that harm system reliability. The proposed DNN architecture includes multi-layer perceptrons with ReLU activation, 3 hidden layers (256, 128, 64 neurons respectively), uses Adam optimizer (learning rate 1e-4) and MSE loss function. I…

Multi-agent learning under uncertainty: Recurrence vs. concentration
Kyriakos Lotidis, Panayotis Mertikopoulos, Nicholas Bambos, Jose Blanchet
https://arxiv.org/abs/2512.08132 https://arxiv.org/pdf/2512.08132 https://arxiv.org/html/2512.08132
arXiv:2512.08132v1 Announce Type: new
Abstract: In this paper, we examine the convergence landscape of multi-agent learning under uncertainty. Specifically, we analyze two stochastic models of regularized learning in continuous games -- one in continuous and one in discrete time with the aim of characterizing the long-run behavior of the induced sequence of play. In stark contrast to deterministic, full-information models of learning (or models with a vanishing learning rate), we show that the resulting dynamics do not converge in general. In lieu of this, we ask instead which actions are played more often in the long run, and by how much. We show that, in strongly monotone games, the dynamics of regularized learning may wander away from equilibrium infinitely often, but they always return to its vicinity in finite time (which we estimate), and their long-run distribution is sharply concentrated around a neighborhood thereof. We quantify the degree of this concentration, and we show that these favorable properties may all break down if the underlying game is not strongly monotone -- underscoring in this way the limits of regularized learning in the presence of persistent randomness and uncertainty.
toXiv_bot_toot

Rate optimal learning of equilibria from data
Till Freihaut, Luca Viano, Emanuele Nevali, Volkan Cevher, Matthieu Geist, Giorgia Ramponi
https://arxiv.org/abs/2510.09325 https:/…

Rate optimal learning of equilibria from data
We close open theoretical gaps in Multi-Agent Imitation Learning (MAIL) by characterizing the limits of non-interactive MAIL and presenting the first interactive algorithm with near-optimal sample complexity. In the non-interactive setting, we prove a statistical lower bound that identifies the all-policy deviation concentrability coefficient as the fundamental complexity measure, and we show that Behavior Cloning (BC) is rate-optimal. For the interactive setting, we introduce a framework that …

Statistical Guarantees for High-Dimensional Stochastic Gradient Descent
Jiaqi Li, Zhipeng Lou, Johannes Schmidt-Hieber, Wei Biao Wu
https://arxiv.org/abs/2510.12013 https://

Statistical Guarantees for High-Dimensional Stochastic Gradient Descent
Stochastic Gradient Descent (SGD) and its Ruppert-Polyak averaged variant (ASGD) lie at the heart of modern large-scale learning, yet their theoretical properties in high-dimensional settings are rarely understood. In this paper, we provide rigorous statistical guarantees for constant learning-rate SGD and ASGD in high-dimensional regimes. Our key innovation is to transfer powerful tools from high-dimensional time series to online learning. Specifically, by viewing SGD as a nonlinear autoregres…

Energy distance and evolution problems: a promising tool for kinetic equations
Gennaro Auricchio, Giuseppe Toscani
https://arxiv.org/abs/2510.09123 https://

Energy distance and evolution problems: a promising tool for kinetic equations
We study the rate of convergence to equilibrium of the solutions to Fokker-Planck type equations with linear drift by means of Cramér and Energy distances, which have been recently widely used in problems related to AI, in particular for tasks related to machine learning. In all cases in which the Fokker-Planck type equations can be treated through these distances, it is shown that the rate of decay is improved with respect to known results which are based on the decay of relative entropy.

Finetune Once: Decoupling General & Domain Learning with Dynamic Boosted Annealing
Yang Tang, Ruijie Liu, Yifan Wang, Shiyu Li, Xi Chen
https://arxiv.org/abs/2509.26242 http…

Finetune Once: Decoupling General & Domain Learning with Dynamic Boosted Annealing
Large language models (LLMs) fine-tuning shows excellent implications. However, vanilla fine-tuning methods often require intricate data mixture and repeated experiments for optimal generalization. To address these challenges and streamline the training process, we propose an efficient and universal solution, Dynamic Boosted Annealing (DBA). We obtain a global gradient through zero-learning-rate training on general data, which is subsequently employed for gradient boosting and dynamic training …

Adaptive Deception Framework with Behavioral Analysis for Enhanced Cybersecurity Defense
Basil Abdullah AL-Zahrani
https://arxiv.org/abs/2510.02424 https://

Adaptive Deception Framework with Behavioral Analysis for Enhanced Cybersecurity Defense
This paper presents CADL (Cognitive-Adaptive Deception Layer), an adaptive deception framework achieving 99.88% detection rate with 0.13% false positive rate on the CICIDS2017 dataset. The framework employs ensemble machine learning (Random Forest, XGBoost, Neural Networks) combined with behavioral profiling to identify and adapt responses to network intrusions. Through a coordinated signal bus architecture, security components share real-time intelligence, enabling collective decision-making. …

Temperature derivative divergence of the electric conductivity and thermal photon emission rate at the critical end point from holography
Yi-Ping Si, Danning Li, Mei Huang
https://arxiv.org/abs/2509.25636

Temperature derivative divergence of the electric conductivity and thermal photon emission rate at the critical end point from holography
The thermal photon emission rate $\frac{dΓ}{dk}$ and DC eletric conductivity $σ_{Q}$ of the strongly coupled quark-gluon plasma (sQGP) are investigated around the critical end point in a $N_f=2+1$ holographic QCD model with parameters obtained from machine-learning. It is found that both thermal photon emission rate and eletric conductivity grow most obviously around $T_c$, which agrees with the previous studies, and the result of eletric conductivity at zero chemical potential resembles the …

False Discovery Rate Control via Bayesian Mirror Statistic
Marco Molinari, Magne Thoresen
https://arxiv.org/abs/2510.00875 https://arxiv.org/pdf/2510.00875…

False Discovery Rate Control via Bayesian Mirror Statistic
Simultaneously performing variable selection and inference in high-dimensional models is an open challenge in statistics and machine learning. The increasing availability of vast amounts of variables requires the adoption of specific statistical procedures to accurately select the most important predictors in a high-dimensional space, while being able to control some form of selection error. In this work we adapt the Mirror Statistic approach to False Discovery Rate (FDR) control into a Bayesia…

Tight Regret Upper and Lower Bounds for Optimistic Hedge in Two-Player Zero-Sum Games
Taira Tsuchiya
https://arxiv.org/abs/2510.11691 https://arxiv.org/pdf…

Tight Regret Upper and Lower Bounds for Optimistic Hedge in Two-Player Zero-Sum Games
In two-player zero-sum games, the learning dynamic based on optimistic Hedge achieves one of the best-known regret upper bounds among strongly-uncoupled learning dynamics. With an appropriately chosen learning rate, the social and individual regrets can be bounded by $O(\log(mn))$ in terms of the numbers of actions $m$ and $n$ of the two players. This study investigates the optimality of the dependence on $m$ and $n$ in the regret of optimistic Hedge. To this end, we begin by refining existing …

Ultra-Reliable Risk-Aggregated Sum Rate Maximization via Model-Aided Deep Learning
Hassaan Hashmi, Spyridon Pougkakiotis, Dionysis Kalogerias
https://arxiv.org/abs/2509.26311 ht…

Ultra-Reliable Risk-Aggregated Sum Rate Maximization via Model-Aided Deep Learning
We consider the problem of maximizing weighted sum rate in a multiple-input single-output (MISO) downlink wireless network with emphasis on user rate reliability. We introduce a novel risk-aggregated formulation of the complex WSR maximization problem, which utilizes the Conditional Value-at-Risk (CVaR) as a functional for enforcing rate (ultra)-reliability over channel fading uncertainty/risk. We establish a WMMSE-like equivalence between the proposed precoding problem and a weighted risk-aver…

Efficient Breast and Ovarian Cancer Classification via ViT-Based Preprocessing and Transfer Learning
Richa Rawat, Faisal Ahmed
https://arxiv.org/abs/2509.18553 https://

Physics-informed learning under mixing: How physical knowledge speeds up learning
Anna Scampicchio, Leonardo F. Toso, Rahel Rickenbach, James Anderson, Melanie N. Zeilinger
https://arxiv.org/abs/2509.24801

Physics-informed learning under mixing: How physical knowledge speeds up learning
A major challenge in physics-informed machine learning is to understand how the incorporation of prior domain knowledge affects learning rates when data are dependent. Focusing on empirical risk minimization with physics-informed regularization, we derive complexity-dependent bounds on the excess risk in probability and in expectation. We prove that, when the physical prior information is aligned, the learning rate improves from the (slow) Sobolev minimax rate to the (fast) optimal i.i.d. one w…

Scale-Invariant Regret Matching and Online Learning with Optimal Convergence: Bridging Theory and Practice in Zero-Sum Games
Brian Hu Zhang, Ioannis Anagnostides, Tuomas Sandholm
https://arxiv.org/abs/2510.04407

Scale-Invariant Regret Matching and Online Learning with Optimal Convergence: Bridging Theory and Practice in Zero-Sum Games
A considerable chasm has been looming for decades between theory and practice in zero-sum game solving through first-order methods. Although a convergence rate of $T^{-1}$ has long been established since Nemirovski's mirror-prox algorithm and Nesterov's excessive gap technique in the early 2000s, the most effective paradigm in practice is *counterfactual regret minimization*, which is based on *regret matching* and its modern variants. In particular, the state of the art across most benchmarks …

On the Adversarial Robustness of Learning-based Conformal Novelty Detection
Daofu Zhang, Mehrdad Pournaderi, Hanne M. Clifford, Yu Xiang, Pramod K. Varshney
https://arxiv.org/abs/2510.00463

On the Adversarial Robustness of Learning-based Conformal Novelty Detection
This paper studies the adversarial robustness of conformal novelty detection. In particular, we focus on AdaDetect, a powerful learning-based framework for novelty detection with finite-sample false discovery rate (FDR) control. While AdaDetect provides rigorous statistical guarantees under benign conditions, its behavior under adversarial perturbations remains unexplored. We first formulate an oracle attack setting that quantifies the worst-case degradation of FDR, deriving an upper bound that…

Why Do We Need Warm-up? A Theoretical Perspective
Foivos Alimisis, Rustem Islamov, Aurelien Lucchi
https://arxiv.org/abs/2510.03164 https://arxiv.org/pdf/2…

Why Do We Need Warm-up? A Theoretical Perspective
Learning rate warm-up - increasing the learning rate at the beginning of training - has become a ubiquitous heuristic in modern deep learning, yet its theoretical foundations remain poorly understood. In this work, we provide a principled explanation for why warm-up improves training. We rely on a generalization of the $(L_0, L_1)$-smoothness condition, which bounds local curvature as a linear function of the loss sub-optimality and exhibits desirable closure properties. We demonstrate both the…

Bayesian Self-Calibration and Parametric Channel Estimation for 6G Antenna Arrays
Patrick H\"odl, Jakob M\"oderl, Erik Leitinger, Klaus Witrisal
https://arxiv.org/abs/2510.11628

Bayesian Self-Calibration and Parametric Channel Estimation for 6G Antenna Arrays
Accurate channel estimation is essential for both high-rate communication and high-precision sensing in 6G wireless systems. However, a major performance limitation arises from calibration mismatches when operating phased-array antennas under real-world conditions. To address this issue, we propose to integrate antenna element self-calibration into a variational sparse Bayesian learning (VSBL) algorithm for parametric channel estimation. We model antenna gain and phase deviations as latent vari…

A Bio-Inspired Minimal Model for Non-Stationary K-Armed Bandits
Krubeal Danieli, Mikkel Elle Lepper{\o}d
https://arxiv.org/abs/2509.22209 https://arxiv.org…

A Bio-Inspired Minimal Model for Non-Stationary K-Armed Bandits
While reinforcement learning algorithms have made significant progress in solving multi-armed bandit problems, they often lack biological plausibility in architecture and dynamics. Here, we propose a bio-inspired neural model based on interacting populations of rate neurons, drawing inspiration from the orbitofrontal cortex and anterior cingulate cortex. Our model reports robust performance across various stochastic bandit problems, matching the effectiveness of standard algorithms such as Thom…

MOCLIP: A Foundation Model for Large-Scale Nanophotonic Inverse Design
S. Rodionov, A. Burguete-Lopez, M. Makarenko, Q. Wang, F. Getman, A. Fratalocchi
https://arxiv.org/abs/2511.18980 https://arxiv.org/pdf/2511.18980 https://arxiv.org/html/2511.18980
arXiv:2511.18980v1 Announce Type: new
Abstract: Foundation models (FM) are transforming artificial intelligence by enabling generalizable, data-efficient solutions across different domains for a broad range of applications. However, the lack of large and diverse datasets limits the development of FM in nanophotonics. This work presents MOCLIP (Metasurface Optics Contrastive Learning Pretrained), a nanophotonic foundation model that integrates metasurface geometry and spectra within a shared latent space. MOCLIP employs contrastive learning to align geometry and spectral representations using an experimentally acquired dataset with a sample density comparable to ImageNet-1K. The study demonstrates MOCLIP inverse design capabilities for high-throughput zero-shot prediction at a rate of 0.2 million samples per second, enabling the design of a full 4-inch wafer populated with high-density metasurfaces in minutes. It also shows generative latent-space optimization reaching 97 percent accuracy. Finally, we introduce an optical information storage concept that uses MOCLIP to achieve a density of 0.1 Gbit per square millimeter at the resolution limit, exceeding commercial optical media by a factor of six. These results position MOCLIP as a scalable and versatile platform for next-generation photonic design and data-driven applications.
toXiv_bot_toot

Generalization Analysis for Classification on Korobov Space
Yuqing Liu
https://arxiv.org/abs/2509.22748 https://arxiv.org/pdf/2509.22748

Generalization Analysis for Classification on Korobov Space
In this paper, the classification algorithm arising from Tikhonov regularization is discussed. The main intention is to derive learning rates for the excess misclassification error according to the convex $η$-norm loss function $ϕ(v)=(1 - v)_{+}^η$, $η\geq1$. Following the argument, the estimation of error under Tsybakov noise conditions is studied. In addition, we propose the rate of $L_p$ approximation of functions from Korobov space $X^{2, p}([-1,1]^{d})$, $1\leq p \leq \infty$, by the s…

When Scores Learn Geometry: Rate Separations under the Manifold Hypothesis
Xiang Li, Zebang Shen, Ya-Ping Hsieh, Niao He
https://arxiv.org/abs/2509.24912 https://

When Scores Learn Geometry: Rate Separations under the Manifold Hypothesis
Score-based methods, such as diffusion models and Bayesian inverse problems, are often interpreted as learning the data distribution in the low-noise limit ($σ\to 0$). In this work, we propose an alternative perspective: their success arises from implicitly learning the data manifold rather than the full distribution. Our claim is based on a novel analysis of scores in the small-$σ$ regime that reveals a sharp separation of scales: information about the data manifold is $Θ(σ^{-2})$ stronger…

Score the Steps, Not Just the Goal: VLM-Based Subgoal Evaluation for Robotic Manipulation
Ramy ElMallah, Krish Chhajer, Chi-Guhn Lee
https://arxiv.org/abs/2509.19524 https://

Score the Steps, Not Just the Goal: VLM-Based Subgoal Evaluation for Robotic Manipulation
Robot learning papers typically report a single binary success rate (SR), which obscures where a policy succeeds or fails along a multi-step manipulation task. We argue that subgoal-level reporting should become routine: for each trajectory, a vector of per-subgoal SRs that makes partial competence visible (e.g., grasp vs. pour). We propose a blueprint for StepEval, a cost-aware plug-in evaluation framework that utilizes vision-language models (VLMs) as automated judges of subgoal outcomes from…

Robust equilibria in continuous games: From strategic to dynamic robustness
Kyriakos Lotidis, Panayotis Mertikopoulos, Nicholas Bambos, Jose Blanchet
https://arxiv.org/abs/2512.08138 https://arxiv.org/pdf/2512.08138 https://arxiv.org/html/2512.08138
arXiv:2512.08138v1 Announce Type: new
Abstract: In this paper, we examine the robustness of Nash equilibria in continuous games, under both strategic and dynamic uncertainty. Starting with the former, we introduce the notion of a robust equilibrium as those equilibria that remain invariant to small -- but otherwise arbitrary -- perturbations to the game's payoff structure, and we provide a crisp geometric characterization thereof. Subsequently, we turn to the question of dynamic robustness, and we examine which equilibria may arise as stable limit points of the dynamics of "follow the regularized leader" (FTRL) in the presence of randomness and uncertainty. Despite their very distinct origins, we establish a structural correspondence between these two notions of robustness: strategic robustness implies dynamic robustness, and, conversely, the requirement of strategic robustness cannot be relaxed if dynamic robustness is to be maintained. Finally, we examine the rate of convergence to robust equilibria as a function of the underlying regularizer, and we show that entropically regularized learning converges at a geometric rate in games with affinely constrained action spaces.
toXiv_bot_toot

Federated Computation of ROC and PR Curves
Xuefeng Xu, Graham Cormode
https://arxiv.org/abs/2510.04979 https://arxiv.org/pdf/2510.04979

Federated Computation of ROC and PR Curves
Receiver Operating Characteristic (ROC) and Precision-Recall (PR) curves are fundamental tools for evaluating machine learning classifiers, offering detailed insights into the trade-offs between true positive rate vs. false positive rate (ROC) or precision vs. recall (PR). However, in Federated Learning (FL) scenarios, where data is distributed across multiple clients, computing these curves is challenging due to privacy and communication constraints. Specifically, the server cannot access raw …

Score the Steps, Not Just the Goal: VLM-Based Subgoal Evaluation for Robotic Manipulation
Ramy ElMallah, Krish Chhajer, Chi-Guhn Lee
https://arxiv.org/abs/2509.19524 https://

Score the Steps, Not Just the Goal: VLM-Based Subgoal Evaluation for Robotic Manipulation
Robot learning papers typically report a single binary success rate (SR), which obscures where a policy succeeds or fails along a multi-step manipulation task. We argue that subgoal-level reporting should become routine: for each trajectory, a vector of per-subgoal SRs that makes partial competence visible (e.g., grasp vs. pour). We propose a blueprint for StepEval, a cost-aware plug-in evaluation framework that utilizes vision-language models (VLMs) as automated judges of subgoal outcomes from…

Improving accuracy in short mortality rate series: Exploring Multi-step Forecasting Approaches in Hybrid Systems
Filipe C. L. Duarte, Paulo S. G. de Mattos Neto, Paulo R. A. Firmino
https://arxiv.org/abs/2509.22395

Improving accuracy in short mortality rate series: Exploring Multi-step Forecasting Approaches in Hybrid Systems
The decline in interest rates and economic stabilization has heightened the importance of accurate mortality rate forecasting, particularly in insurance and pension markets. Multi-step-ahead predictions are crucial for public health, demographic planning, and insurance risk assessments; however, they face challenges when data are limited. Hybrid systems that combine statistical and Machine Learning (ML) models offer a promising solution for handling both linear and nonlinear patterns. This stud…

Data-Driven Reconstruction of Significant Wave Heights from Sparse Observations
Hongyuan Shi, Yilin Zhai, Ping Dong, Zaijin You, Chao Zhan, Qing Wang
https://arxiv.org/abs/2509.19384

Data-Driven Reconstruction of Significant Wave Heights from Sparse Observations
Reconstructing high-resolution regional significant wave height fields from sparse and uneven buoy observations remains a core challenge for ocean monitoring and risk-aware operations. We introduce AUWave, a hybrid deep learning framework that fuses a station-wise sequence encoder (MLP) with a multi-scale U-Net enhanced by a bottleneck self-attention layer to recover 32$\times$32 regional SWH fields. A systematic Bayesian hyperparameter search with Optuna identifies the learning rate as the dom…

Effective continuous equations for adaptive SGD: a stochastic analysis view
Luca Callisti, Marco Romito, Francesco Triggiano
https://arxiv.org/abs/2509.21614 https://

Effective continuous equations for adaptive SGD: a stochastic analysis view
We present a theoretical analysis of some popular adaptive Stochastic Gradient Descent (SGD) methods in the small learning rate regime. Using the stochastic modified equations framework introduced by Li et al., we derive effective continuous stochastic dynamics for these methods. Our key contribution is that sampling-induced noise in SGD manifests in the limit as independent Brownian motions driving the parameter and gradient second momentum evolutions. Furthermore, extending the approach of Ma…

Group Relative Policy Optimization for Text-to-Speech with Large Language Models
Chang Liu, Ya-Jun Hu, Ying-Ying Gao, Shi-Lei Zhang, Zhen-Hua Ling
https://arxiv.org/abs/2509.18798

Group Relative Policy Optimization for Text-to-Speech with Large Language Models
This paper proposes a GRPO-based approach to enhance the performance of large language model (LLM)-based text-to-speech (TTS) models by deriving rewards from an off-the-shelf automatic speech recognition (ASR) model. Compared to previous reinforcement learning methods for LLM-based TTS, our method requires no dedicated model for reward computation or training. Moreover, we design a composite reward function that combines character error rate (CER) with negative log-likelihood (NLL) obtained fro…

Efficient Hyperparameter Tuning via Trajectory Invariance Principle
Bingrui Li, Jiaxin Wen, Zhanpeng Zhou, Jun Zhu, Jianfei Chen
https://arxiv.org/abs/2509.25049 https://…

Efficient Hyperparameter Tuning via Trajectory Invariance Principle
As hyperparameter tuning becomes increasingly costly at scale, efficient tuning methods are essential. Yet principles for guiding hyperparameter tuning remain limited. In this work, we seek to establish such principles by considering a broad range of hyperparameters, including batch size, learning rate, and weight decay. We identify a phenomenon we call trajectory invariance, where pre-training loss curves, gradient noise, and gradient norm exhibit invariance--closely overlapping--with respect …

Scaling with Collapse: Efficient and Predictable Training of LLM Families
Shane Bergsma, Bin Claire Zhang, Nolan Dey, Shaheer Muhammad, Gurpreet Gosal, Joel Hestness
https://arxiv.org/abs/2509.25087

Scaling with Collapse: Efficient and Predictable Training of LLM Families
Effective LLM training relies on *consistency*, meaning that key quantities -- such as final losses and optimal hyperparameters -- scale predictably across model sizes. Qiu et al. (2025) recently showed that this consistency extends beyond scalars: whole training loss curves can *collapse* onto a universal trajectory after a simple normalization. What remains unclear is whether this phenomenon holds for LLM families trained under *practical scaling recipes*, where width, depth, learning rate, b…