Tootfinder

This https://arxiv.org/abs/2312.02572 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDB_…

Efficient Enumeration of Recursive Plans in Transformation-based Query Optimizers
Query optimizers built on the transformation-based Volcano/Cascades framework are used in many database systems. Transformations proposed earlier on the logical query dag (LQDAG) data structure, which is key in such a framework, focus only on recursion-free queries. In this paper, we propose the recursive logical query dag (RLQDAG) which extends the LQDAG with the ability to capture and transform recursive queries, leveraging recent developments in recursive relational algebra. Specifically, th…

SGD with Partial Hessian for Deep Neural Networks Optimization
Ying Sun, Hongwei Yong, Lei Zhang
https://arxiv.org/abs/2403.02681 https://

SGD with Partial Hessian for Deep Neural Networks Optimization
Due to the effectiveness of second-order algorithms in solving classical optimization problems, designing second-order optimizers to train deep neural networks (DNNs) has attracted much research interest in recent years. However, because of the very high dimension of intermediate features in DNNs, it is difficult to directly compute and store the Hessian matrix for network optimization. Most of the previous second-order methods approximate the Hessian information imprecisely, resulting in unsta…

This https://arxiv.org/abs/2011.01718 has been replaced.
link: https://scholar.google.com/scholar?q=a

Multi-Iteration Stochastic Optimizers
We here introduce Multi-Iteration Stochastic Optimizers, a novel class of first-order stochastic optimizers where the relative $L^2$ error is estimated and controlled using successive control variates along the path of iterations. By exploiting the correlation between iterates, control variates may reduce the estimator's variance so that an accurate estimation of the mean gradient becomes computationally affordable. We name the estimator of the mean gradient Multi-Iteration stochastiC Estimator…

This https://arxiv.org/abs/2312.02572 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDB_…

Efficient Enumeration of Recursive Plans in Transformation-based Query Optimizers
Query optimizers built on the transformation-based Volcano/Cascades framework are used in many database systems. Transformations proposed earlier on the logical query dag (LQDAG) data structure, which is key in such a framework, focus only on recursion-free queries. In this paper, we propose the recursive logical query dag (RLQDAG) which extends the LQDAG with the ability to capture and transform recursive queries, leveraging recent developments in recursive relational algebra. Specifically, th…

When you set your paperclip optimizers to optimize for "engagement," this is the obvious, inevitable result.
https://www.abc.net.au/news/science/2024-02-28/twitter-x-fighting-bot-problem-as-ai-spam-fl…

Twitter is becoming a 'ghost town' of bots as AI-generated spam content floods the internet
The internet is filling up with machine-generated "zombie content" designed to game algorithms and scam humans. Experts call it the "great AI flood".

Go Beyond Black-box Policies: Rethinking the Design of Learning Agent for Interpretable and Verifiable HVAC Control
Zhiyu An, Xianzhong Ding, Wan Du
https://arxiv.org/abs/2403.00172

Go Beyond Black-box Policies: Rethinking the Design of Learning Agent for Interpretable and Verifiable HVAC Control
Recent research has shown the potential of Model-based Reinforcement Learning (MBRL) to enhance energy efficiency of Heating, Ventilation, and Air Conditioning (HVAC) systems. However, existing methods rely on black-box thermal dynamics models and stochastic optimizers, lacking reliability guarantees and posing risks to occupant health. In this work, we overcome the reliability bottleneck by redesigning HVAC controllers using decision trees extracted from existing thermal dynamics models and hi…

This https://arxiv.org/abs/2310.19046 has been replaced.
link: https://scholar.google.com/scholar?q=a

Large Language Models as Evolutionary Optimizers
Evolutionary algorithms (EAs) have achieved remarkable success in tackling complex combinatorial optimization problems. However, EAs often demand carefully-designed operators with the aid of domain expertise to achieve satisfactory performance. In this work, we present the first study on large language models (LLMs) as evolutionary combinatorial optimizers. The main advantage is that it requires minimal domain knowledge and human efforts, as well as no additional training of the model. This app…

Proteus: Preserving Model Confidentiality during Graph Optimizations
Yubo Gao, Maryam Haghifam, Christina Giannoula, Renbo Tu, Gennady Pekhimenko, Nandita Vijaykumar
https://arxiv.org/abs/2404.12512

Proteus: Preserving Model Confidentiality during Graph Optimizations
Deep learning (DL) models have revolutionized numerous domains, yet optimizing them for computational efficiency remains a challenging endeavor. Development of new DL models typically involves two parties: the model developers and performance optimizers. The collaboration between the parties often necessitates the model developers exposing the model architecture and computational graph to the optimizers. However, this exposure is undesirable since the model architecture is an important intellec…

LangGPT: Rethinking Structured Reusable Prompt Design Framework for LLMs from the Programming Language
Ming Wang, Yuanzhong Liu, Xiaoming Zhang, Songlian Li, Yijie Huang, Chi Zhang, Daling Wang, Shi Feng, Jigang Li
https://arxiv.org/abs/2402.16929

LangGPT: Rethinking Structured Reusable Prompt Design Framework for LLMs from the Programming Language
LLMs have demonstrated commendable performance across diverse domains. Nevertheless, formulating high-quality prompts to effectively instruct LLMs poses a challenge for non-AI experts. Existing research in prompt engineering suggests somewhat fragmented optimization principles and designs empirically dependent prompt optimizers. Unfortunately, these endeavors lack a structured design template, incurring high learning costs and resulting in low reusability. Inspired by structured reusable progra…

This https://arxiv.org/abs/2308.07917 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Degenerate Stability of the Caffarelli-Kohn-Nirenberg Inequality along the Felli-Schneider Curve
We show that the Caffarelli-Kohn-Nirenberg (CKN) inequality holds with a remainder term that is quartic in the distance to the set of optimizers for the full parameter range of the Felli-Schneider (FS) curve. The fourth power is best possible. This is due to the presence of non-trivial zero modes of the Hessian of the deficit functional along the FS-curve. Following an iterated Bianchi-Egnell strategy, the heart of our proof is verifying a `secondary non-degeneracy condition'. Our result comple…

This https://arxiv.org/abs/2402.11858 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Stochastic Hessian Fittings on Lie Groups
This paper studies the fitting of Hessian or its inverse for stochastic optimizations using a Hessian fitting criterion from the preconditioned stochastic gradient descent (PSGD) method, which is intimately related to many commonly used second order and adaptive gradient optimizers, e.g., BFGS, Gaussian-Newton and natural gradient descent, AdaGrad, etc. Our analyses reveal the efficiency and reliability differences among a wide range of preconditioner fitting methods, from closed-form to iterat…

Learning in PINNs: Phase transition, total diffusion, and generalization
Sokratis J. Anagnostopoulos, Juan Diego Toscano, Nikolaos Stergiopulos, George Em Karniadakis
https://arxiv.org/abs/2403.18494

Learning in PINNs: Phase transition, total diffusion, and generalization
We investigate the learning dynamics of fully-connected neural networks through the lens of gradient signal-to-noise ratio (SNR), examining the behavior of first-order optimizers like Adam in non-convex objectives. By interpreting the drift/diffusion phases in the information bottleneck theory, focusing on gradient homogeneity, we identify a third phase termed ``total diffusion", characterized by equilibrium in the learning rates and homogeneous gradients. This phase is marked by an abrupt SNR …

This https://arxiv.org/abs/2108.12981 has been replaced.
link: https://scholar.google.com/scholar?q=a

The ensmallen library for flexible numerical optimization
We overview the ensmallen numerical optimization library, which provides a flexible C++ framework for mathematical optimization of user-supplied objective functions. Many types of objective functions are supported, including general, differentiable, separable, constrained, and categorical. A diverse set of pre-built optimizers is provided, including Quasi-Newton optimizers and many variants of Stochastic Gradient Descent. The underlying framework facilitates the implementation of new optimizers…

This https://arxiv.org/abs/2309.08770 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csRO_…

Constrained Bimanual Planning with Analytic Inverse Kinematics
In order for a bimanual robot to manipulate an object that is held by both hands, it must construct motion plans such that the transformation between its end effectors remains fixed. This amounts to complicated nonlinear equality constraints in the configuration space, which are difficult for trajectory optimizers. In addition, the set of feasible configurations becomes a measure zero set, which presents a challenge to sampling-based motion planners. We leverage an analytic solution to the inve…

Adapting Multi-objectivized Software Configuration Tuning
Tao Chen, Miqing Li
https://arxiv.org/abs/2404.04744 https://arxiv.org/pdf/…

Adapting Multi-objectivized Software Configuration Tuning
When tuning software configuration for better performance (e.g., latency or throughput), an important issue that many optimizers face is the presence of local optimum traps, compounded by a highly rugged configuration landscape and expensive measurements. To mitigate these issues, a recent effort has shifted to focus on the level of optimization model (called meta multi-objectivization or MMO) instead of designing better optimizers as in traditional methods. This is done by using an auxiliary p…

This https://arxiv.org/abs/2309.01551 has been replaced.
link: https://scholar.google.com/scholar?q=a

Is Your Learned Query Optimizer Behaving As You Expect? A Machine Learning Perspective
The current boom of learned query optimizers (LQO) can be explained not only by the general continuous improvement of deep learning (DL) methods but also by the straightforward formulation of a query optimization problem (QOP) as a machine learning (ML) one. The idea is often to replace dynamic programming approaches, widespread for solving QOP, with more powerful methods such as reinforcement learning. However, such a rapid "game change" in the field of QOP could not pass without consequences …

Neural Optimizer Equation, Decay Function, and Learning Rate Schedule Joint Evolution
Brandon Morgan, Dean Hougen
https://arxiv.org/abs/2404.06679 https://…

Neural Optimizer Equation, Decay Function, and Learning Rate Schedule Joint Evolution
A major contributor to the quality of a deep learning model is the selection of the optimizer. We propose a new dual-joint search space in the realm of neural optimizer search (NOS), along with an integrity check, to automate the process of finding deep learning optimizers. Our dual-joint search space simultaneously allows for the optimization of not only the update equation, but also internal decay functions and learning rate schedules for optimizers. We search the space using our proposed mut…

This https://arxiv.org/abs/2309.01551 has been replaced.
link: https://scholar.google.com/scholar?q=a

Is Your Learned Query Optimizer Behaving As You Expect? A Machine Learning Perspective
The current boom of learned query optimizers (LQO) can be explained not only by the general continuous improvement of deep learning (DL) methods but also by the straightforward formulation of a query optimization problem (QOP) as a machine learning (ML) one. The idea is often to replace dynamic programming approaches, widespread for solving QOP, with more powerful methods such as reinforcement learning. However, such a rapid "game change" in the field of QOP could not pass without consequences …

This https://arxiv.org/abs/2310.06650 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

A Parallelized, Adam-Based Solver for Reserve and Security Constrained AC Unit Commitment
Power system optimization problems which include the nonlinear AC power flow equations require powerful and robust numerical solution algorithms. Within this sub-field of nonlinear optimization, interior point methods have come to dominate the solver landscape. Over the last decade, however, a number of efficient numerical optimizers have emerged from the field of Machine Learning (ML). One algorithm in particular, Adam, has become the optimizer-of-choice for a massive percentage of ML training…

This https://arxiv.org/abs/2402.11858 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Stochastic Hessian Fittings with Lie Groups
This paper studies the fitting of Hessian or its inverse for stochastic optimizations using a Hessian fitting criterion from the preconditioned stochastic gradient descent (PSGD) method, which is intimately related to many commonly used second order and adaptive gradient optimizers, e.g., BFGS, Gaussian-Newton and natural gradient descent, AdaGrad, etc. Our analyses reveal the efficiency and reliability differences among a wide range of preconditioner fitting methods, from closed-form to iterat…

Discrete functional inequalities on lattice graphs
Shubham Gupta
https://arxiv.org/abs/2403.10270 https://arxiv.org/pdf/2403.10270

Discrete functional inequalities on lattice graphs
In this thesis, we study problems at the interface of analysis and discrete mathematics. We discuss analogues of well known Hardy-type inequalities and Rearrangement inequalities on the lattice graphs $\mathbb{Z}^d$, with a particular focus on behaviour of sharp constants and optimizers.In the first half of the thesis, we analyse Hardy inequalities on $\mathbb{Z}^d$, first for $d=1$ and then for $d \geq 3$. We prove a sharp weighted Hardy inequality on integers with power weights of the form $n…

This https://arxiv.org/abs/2301.13646 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Nonlinear Optimization Filters for Stochastic Time-Varying Convex Optimization
We look at a stochastic time-varying optimization problem and we formulate online algorithms to find and track its optimizers in expectation. The algorithms are derived from the intuition that standard prediction and correction steps can be seen as a nonlinear dynamical system and a measurement equation, respectively, yielding the notion of nonlinear filter design. The optimization algorithms are then based on an extended Kalman filter in the unconstrained case, and on a bilinear matrix inequal…

Evolving Benchmark Functions to Compare Evolutionary Algorithms via Genetic Programming
Yifan He, Claus Aranha
https://arxiv.org/abs/2403.14146 https://

Evolving Benchmark Functions to Compare Evolutionary Algorithms via Genetic Programming
In this study, we use Genetic Programming (GP) to compose new optimization benchmark functions. Optimization benchmarks have the important role of showing the differences between evolutionary algorithms, making it possible for further analysis and comparisons. We show that the benchmarks generated by GP are able to differentiate algorithms better than human-made benchmark functions. The fitness measure of the GP is the Wasserstein distance of the solutions found by a pair of optimizers. Additio…

This https://arxiv.org/abs/2402.11858 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Stochastic Hessian Fittings with Lie Groups
This paper studies the fitting of Hessian or its inverse for stochastic optimizations using a Hessian fitting criterion from the preconditioned stochastic gradient descent (PSGD) method, which is intimately related to many commonly used second order and adaptive gradient optimizers, e.g., BFGS, Gaussian-Newton and natural gradient descent, AdaGrad, etc. Our analyses reveal the efficiency and reliability differences among a wide range of preconditioner fitting methods, from closed-form to iterat…

Auto-configuring Exploration-Exploitation Tradeoff in Evolutionary Computation via Deep Reinforcement Learning
Zeyuan Ma, Jiacheng Chen, Hongshu Guo, Yining Ma, Yue-Jiao Gong
https://arxiv.org/abs/2404.08239

Auto-configuring Exploration-Exploitation Tradeoff in Evolutionary Computation via Deep Reinforcement Learning
Evolutionary computation (EC) algorithms, renowned as powerful black-box optimizers, leverage a group of individuals to cooperatively search for the optimum. The exploration-exploitation tradeoff (EET) plays a crucial role in EC, which, however, has traditionally been governed by manually designed rules. In this paper, we propose a deep reinforcement learning-based framework that autonomously configures and adapts the EET throughout the EC search process. The framework allows different individu…