Tootfinder

Optimal and Practical Batched Linear Bandit Algorithm
Sanghoon Yu, Min-hwan Oh
https://arxiv.org/abs/2507.08438 https://arxiv.org/pdf…

Optimal and Practical Batched Linear Bandit Algorithm
We study the linear bandit problem under limited adaptivity, known as the batched linear bandit. While existing approaches can achieve near-optimal regret in theory, they are often computationally prohibitive or underperform in practice. We propose \texttt{BLAE}, a novel batched algorithm that integrates arm elimination with regularized G-optimal design, achieving the minimax optimal regret (up to logarithmic factors in $T$) in both large-$K$ and small-$K$ regimes for the first time, while usin…

Tree-Structured Parzen Estimator Can Solve Black-Box Combinatorial Optimization More Efficiently
Kenshin Abe, Yunzhuo Wang, Shuhei Watanabe
https://arxiv.org/abs/2507.08053 https://arxiv.org/pdf/2507.08053 https://arxiv.org/html/2507.08053
arXiv:2507.08053v1 Announce Type: new
Abstract: Tree-structured Parzen estimator (TPE) is a versatile hyperparameter optimization (HPO) method supported by popular HPO tools. Since these HPO tools have been developed in line with the trend of deep learning (DL), the problem setups often used in the DL domain have been discussed for TPE such as multi-objective optimization and multi-fidelity optimization. However, the practical applications of HPO are not limited to DL, and black-box combinatorial optimization is actively utilized in some domains, e.g., chemistry and biology. As combinatorial optimization has been an untouched, yet very important, topic in TPE, we propose an efficient combinatorial optimization algorithm for TPE. In this paper, we first generalize the categorical kernel with the numerical kernel in TPE, enabling us to introduce a distance structure to the categorical kernel. Then we discuss modifications for the newly developed kernel to handle a large combinatorial search space. These modifications reduce the time complexity of the kernel calculation with respect to the size of a combinatorial search space. In the experiments using synthetic problems, we verified that our proposed method identifies better solutions with fewer evaluations than the original TPE. Our algorithm is available in Optuna, an open-source framework for HPO.
toXiv_bot_toot

heise | Kleiner Haushalt, kleiner Verbrauch: Stromkostenoptimierung für Single-Haushalte
Es gab in Deutschland noch nie so viele Single-Haushalte wie heute. Deshalb schauen wir uns an, wie diese kleinen Haushalte ihre Stromkosten optimieren können.

Multi-Timescale Dynamics Model Bayesian Optimization for Plasma Stabilization in Tokamaks
Rohit Sonker, Alexandre Capone, Andrew Rothstein, Hiro Josep Farre Kaga, Egemen Kolemen, Jeff Schneider
https://arxiv.org/abs/2506.10287

Multi-Timescale Dynamics Model Bayesian Optimization for Plasma Stabilization in Tokamaks
Machine learning algorithms often struggle to control complex real-world systems. In the case of nuclear fusion, these challenges are exacerbated, as the dynamics are notoriously complex, data is poor, hardware is subject to failures, and experiments often affect dynamics beyond the experiment's duration. Existing tools like reinforcement learning, supervised learning, and Bayesian optimization address some of these challenges but fail to provide a comprehensive solution. To overcome these limi…

AdaptiveLLM: A Framework for Selecting Optimal Cost-Efficient LLM for Code-Generation Based on CoT Length
Junhang Cheng, Fang Liu, Chengru Wu, Li Zhang
https://arxiv.org/abs/2506.10525

AdaptiveLLM: A Framework for Selecting Optimal Cost-Efficient LLM for Code-Generation Based on CoT Length
While Large Language Models (LLMs) have significantly advanced code generation efficiency, they face inherent challenges in balancing performance and inference costs across diverse programming tasks. Dynamically selecting the optimal LLM based on task difficulty and resource constraints offers a promising approach to achieve an optimal balance between efficiency and performance. However, existing model selection methods are resource-intensive and often neglect cost efficiency. Moreover, these a…

Towards solving large QUBO problems using quantum algorithms: improving the LogQ scheme
Yagnik Chatterjee, J\'er\'emie Messud
https://arxiv.org/abs/2507.08489

Towards solving large QUBO problems using quantum algorithms: improving the LogQ scheme
The LogQ algorithm encodes Quadratic Unconstrained Binary Optimization (QUBO) problems with exponentially fewer qubits than the Quantum Approximate Optimization Algorithm (QAOA). The advantages of conventional LogQ are accompanied by a challenge related to the optimization of its free parameters, which requires the usage of resource intensive evolutionary or even global optimization algorithms. We propose a new LogQ parameterization that can be optimized with a gradient-inspired method, which i…

This session has a plot twist at the end that qualifies as a cinematic masterpiece of the year:
https://developer.apple.com/videos/play/wwdc2025/308

Optimize CPU performance with Instruments - WWDC25 - Videos - Apple Developer
Learn how to optimize your app for Apple silicon with two new hardware-assisted tools in Instruments. We'll start by covering how to...

Learning-Based Stable Optimal Control for Infinite-Time Nonlinear Regulation Problems
Han Wang, Di Wu, Lin Cheng, Shengping Gong, Xu Huang
https://arxiv.org/abs/2506.10291

Learning-Based Stable Optimal Control for Infinite-Time Nonlinear Regulation Problems
Infinite-time nonlinear optimal regulation control is widely utilized in aerospace engineering as a systematic method for synthesizing stable controllers. However, conventional methods often rely on linearization hypothesis, while recent learning-based approaches rarely consider stability guarantees. This paper proposes a learning-based framework to learn a stable optimal controller for nonlinear optimal regulation problems. First, leveraging the equivalence between Pontryagin Maximum Principle…

PDE-aware Optimizer for Physics-informed Neural Networks
Hardik Shukla, Manurag Khullar, Vismay Churiwala
https://arxiv.org/abs/2507.08118 https://arxiv.org/pdf/2507.08118 https://arxiv.org/html/2507.08118
arXiv:2507.08118v1 Announce Type: new
Abstract: Physics-Informed Neural Networks (PINNs) have emerged as a powerful framework for solving partial differential equations (PDEs) by embedding physical constraints into the loss function. However, standard optimizers such as Adam often struggle to balance competing loss terms, particularly in stiff or ill-conditioned systems. In this work, we propose a PDE-aware optimizer that adapts parameter updates based on the variance of per-sample PDE residual gradients. This method addresses gradient misalignment without incurring the heavy computational costs of second-order optimizers such as SOAP. We benchmark the PDE-aware optimizer against Adam and SOAP on 1D Burgers', Allen-Cahn and Korteweg-de Vries(KdV) equations. Across both PDEs, the PDE-aware optimizer achieves smoother convergence and lower absolute errors, particularly in regions with sharp gradients. Our results demonstrate the effectiveness of PDE residual-aware adaptivity in enhancing stability in PINNs training. While promising, further scaling on larger architectures and hardware accelerators remains an important direction for future research.
toXiv_bot_toot

Low-rank Momentum Factorization for Memory Efficient Training
Pouria Mahdavinia, Mehrdad Mahdavi
https://arxiv.org/abs/2507.08091 https://arxiv.org/pdf/2507.08091 https://arxiv.org/html/2507.08091
arXiv:2507.08091v1 Announce Type: new
Abstract: Fine-tuning large foundation models presents significant memory challenges due to stateful optimizers like AdamW, often requiring several times more GPU memory than inference. While memory-efficient methods like parameter-efficient fine-tuning (e.g., LoRA) and optimizer state compression exist, recent approaches like GaLore bridge these by using low-rank gradient projections and subspace moment accumulation. However, such methods may struggle with fixed subspaces or computationally costly offline resampling (e.g., requiring full-matrix SVDs). We propose Momentum Factorized SGD (MoFaSGD), which maintains a dynamically updated low-rank SVD representation of the first-order momentum, closely approximating its full-rank counterpart throughout training. This factorization enables a memory-efficient fine-tuning method that adaptively updates the optimization subspace at each iteration. Crucially, MoFaSGD leverages the computed low-rank momentum factors to perform efficient spectrally normalized updates, offering an alternative to subspace moment accumulation. We establish theoretical convergence guarantees for MoFaSGD, proving it achieves an optimal rate for non-convex stochastic optimization under standard assumptions. Empirically, we demonstrate MoFaSGD's effectiveness on large language model alignment benchmarks, achieving a competitive trade-off between memory reduction (comparable to LoRA) and performance compared to state-of-the-art low-rank optimization methods. Our implementation is available at https://github.com/pmahdavi/MoFaSGD.
toXiv_bot_toot