Tootfinder

Bridging Offline and Online Reinforcement Learning for LLMs
Jack Lanchantin, Angelica Chen, Janice Lan, Xian Li, Swarnadeep Saha, Tianlu Wang, Jing Xu, Ping Yu, Weizhe Yuan, Jason E Weston, Sainbayar Sukhbaatar, Ilia Kulikov
https://arxiv.org/abs/2506.21495 https://arxiv.org/pdf/2506.21495 https://arxiv.org/html/2506.21495
arXiv:2506.21495v1 Announce Type: new
Abstract: We investigate the effectiveness of reinforcement learning methods for finetuning large language models when transitioning from offline to semi-online to fully online regimes for both verifiable and non-verifiable tasks. Our experiments cover training on verifiable math as well as non-verifiable instruction following with a set of benchmark evaluations for both. Across these settings, we extensively compare online and semi-online Direct Preference Optimization and Group Reward Policy Optimization objectives, and surprisingly find similar performance and convergence between these variants, which all strongly outperform offline methods. We provide a detailed analysis of the training dynamics and hyperparameter selection strategies to achieve optimal results. Finally, we show that multi-tasking with verifiable and non-verifiable rewards jointly yields improved performance across both task types.
toXiv_bot_toot

carps: A Framework for Comparing N Hyperparameter Optimizers on M Benchmarks
Carolin Benjamins, Helena Graf, Sarah Segel, Difan Deng, Tim Ruhkopf, Leona Hennig, Soham Basu, Neeratyoy Mallik, Edward Bergman, Deyao Chen, Fran\c{c}ois Cl\'ement, Matthias Feurer, Katharina Eggensperger, Frank Hutter, Carola Doerr, Marius Lindauer
https://

carps: A Framework for Comparing N Hyperparameter Optimizers on M Benchmarks
Hyperparameter Optimization (HPO) is crucial to develop well-performing machine learning models. In order to ease prototyping and benchmarking of HPO methods, we propose carps, a benchmark framework for Comprehensive Automated Research Performance Studies allowing to evaluate N optimizers on M benchmark tasks. In this first release of carps, we focus on the four most important types of HPO task types: blackbox, multi-fidelity, multi-objective and multi-fidelity-multi-objective. With 3 336 tasks…

This https://arxiv.org/abs/2412.06481 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

DeePC-Hunt: Data-enabled Predictive Control Hyperparameter Tuning via Differentiable Optimization
This paper introduces Data-enabled Predictive Control Hyperparameter Tuning via Differentiable Optimization (DeePC-Hunt), a backpropagation-based method for automatic hyperparameter tuning of the DeePC algorithm. The necessity for such a method arises from the importance of hyperparameter selection to achieve satisfactory closed-loop DeePC performance. The standard methods for hyperparameter selection are to either optimize the open-loop performance, or use manual guess-and-check. Optimizing th…

Great Restraining Wall in Multidimentional Collective Variable Space
Zhijun Pan, Maodong Li, Dechin Chen, Yi Isaac Yang
https://arxiv.org/abs/2506.17043 ht…

Great Restraining Wall in Multidimentional Collective Variable Space
Enhanced sampling methods are pivotal for exploring rare events in molecular dynamics (MD), yet face challenges in high-dimensional collective variable (CV) spaces where exhaustive sampling becomes computationally prohibitive. While techniques like metadynamics (MetaD) and path-CV enable targeted free energy surface (FES) reconstruction, they often struggle with confinement stability, hyperparameter sensitivity, and geometric flexibility. This work introduces the Great Restraining Wall (GW) met…

This https://arxiv.org/abs/2412.06481 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

DeePC-Hunt: Data-enabled Predictive Control Hyperparameter Tuning via Differentiable Optimization
This paper introduces Data-enabled Predictive Control Hyperparameter Tuning via Differentiable Optimization (DeePC-Hunt), a backpropagation-based method for automatic hyperparameter tuning of the DeePC algorithm. The necessity for such a method arises from the importance of hyperparameter selection to achieve satisfactory closed-loop DeePC performance. The standard methods for hyperparameter selection are to either optimize the open-loop performance, or use manual guess-and-check. Optimizing th…

A Practical Guide to Tuning Spiking Neuronal Dynamics
William Gebhardt, Alexander G. Ororbia, Nathan McDonald, Clare Thiem, Jack Lombardi
https://arxiv.org/abs/2506.08138

A Practical Guide to Tuning Spiking Neuronal Dynamics
In this work, we examine fundamental elements of spiking neural networks (SNNs) as well as how to tune them. Concretely, we focus on two different foundational neuronal units utilized in SNNs -- the leaky integrate-and-fire (LIF) and the resonate-and-fire (RAF) neuron. We explore key equations and how hyperparameter values affect behavior. Beyond hyperparameters, we discuss other important design elements of SNNs -- the choice of input encoding and the setup for excitatory-inhibitory population…

This https://arxiv.org/abs/2502.06044 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Scalable Differentially Private Bayesian Optimization
In recent years, there has been much work on scaling Bayesian Optimization to high-dimensional problems, for example hyperparameter tuning in large machine learning models. These scalable methods have been successful, finding high objective values much more quickly than traditional global Bayesian Optimization or random search-based methods. At the same time, these large models often use sensitive data, but preservation of Differential Privacy has not scaled alongside these modern Bayesian Opti…

Vid2Sim: Generalizable, Video-based Reconstruction of Appearance, Geometry and Physics for Mesh-free Simulation
Chuhao Chen, Zhiyang Dou, Chen Wang, Yiming Huang, Anjun Chen, Qiao Feng, Jiatao Gu, Lingjie Liu
https://arxiv.org/abs/2506.06440

Vid2Sim: Generalizable, Video-based Reconstruction of Appearance, Geometry and Physics for Mesh-free Simulation
Faithfully reconstructing textured shapes and physical properties from videos presents an intriguing yet challenging problem. Significant efforts have been dedicated to advancing such a system identification problem in this area. Previous methods often rely on heavy optimization pipelines with a differentiable simulator and renderer to estimate physical parameters. However, these approaches frequently necessitate extensive hyperparameter tuning for each scene and involve a costly optimization p…

This https://arxiv.org/abs/2506.05673 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

Peer-Ranked Precision: Creating a Foundational Dataset for Fine-Tuning Vision Models from DataSeeds' Annotated Imagery
The development of modern Artificial Intelligence (AI) models, particularly diffusion-based models employed in computer vision and image generation tasks, is undergoing a paradigmatic shift in development methodologies. Traditionally dominated by a "Model Centric" approach, in which performance gains were primarily pursued through increasingly complex model architectures and hyperparameter optimization, the field is now recognizing a more nuanced "Data-Centric" approach. This emergent framework…

On the Effectiveness of Classical Regression Methods for Optimal Switching Problems
Martin Andersson, Benny Avelin, Marcus Olofsson
https://arxiv.org/abs/2506.15436

On the Effectiveness of Classical Regression Methods for Optimal Switching Problems
Simple regression methods provide robust, near-optimal solutions for optimal switching problems in dimensions ranging from 1 to 50. While the theory requires solving intractable PDE systems, the Longstaff-Schwartz algorithm with classical approaches like $k$-NN achieves excellent switching decisions without extensive hyperparameter tuning. Testing eight regression approaches on four benchmark problems, we find that simple methods maintain stable performance across diverse problem characteristic…

This https://arxiv.org/abs/2503.22733 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

RBFleX-NAS: Training-Free Neural Architecture Search Using Radial Basis Function Kernel and Hyperparameter Detection
Neural Architecture Search (NAS) is an automated technique to design optimal neural network architectures for a specific workload. Conventionally, evaluating candidate networks in NAS involves extensive training, which requires significant time and computational resources. To address this, training-free NAS has been proposed to expedite network evaluation with minimal search time. However, state-of-the-art training-free NAS algorithms struggle to precisely distinguish well-performing networks f…

This https://arxiv.org/abs/2505.00812 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

Handling Label Noise via Instance-Level Difficulty Modeling and Dynamic Optimization
Recent studies indicate that deep neural networks degrade in generalization performance under noisy supervision. Existing methods focus on isolating clean subsets or correcting noisy labels, facing limitations such as high computational costs, heavy hyperparameter tuning process, and coarse-grained optimization. To address these challenges, we propose a novel two-stage noisy learning framework that enables instance-level optimization through a dynamically weighted loss function, avoiding hyperp…

Fine-tuning for Data-enabled Predictive Control of Noisy Systems by Reinforcement Learning
Jinbao Wang, Shiliang Zhang, Jun Liu, Xuehui Ma, Haolin Liu
https://arxiv.org/abs/2505.24572

Fine-tuning for Data-enabled Predictive Control of Noisy Systems by Reinforcement Learning
Data-enabled predictive control (DeePC) leverages system measurements in characterizing system dynamics for optimal control. The performance of DeePC relies on optimizing its hyperparameters, especially in noisy systems where the optimal hyperparameters adapt over time. Existing hyperparameter tuning approaches for DeePC are more than often computationally inefficient or overly conservative. This paper proposes an adaptive DeePC where we guide its hyperparameters adaption through reinforcement …