Tootfinder

Low-rank Momentum Factorization for Memory Efficient Training
Pouria Mahdavinia, Mehrdad Mahdavi
https://arxiv.org/abs/2507.08091 https://arxiv.org/pdf/2507.08091 https://arxiv.org/html/2507.08091
arXiv:2507.08091v1 Announce Type: new
Abstract: Fine-tuning large foundation models presents significant memory challenges due to stateful optimizers like AdamW, often requiring several times more GPU memory than inference. While memory-efficient methods like parameter-efficient fine-tuning (e.g., LoRA) and optimizer state compression exist, recent approaches like GaLore bridge these by using low-rank gradient projections and subspace moment accumulation. However, such methods may struggle with fixed subspaces or computationally costly offline resampling (e.g., requiring full-matrix SVDs). We propose Momentum Factorized SGD (MoFaSGD), which maintains a dynamically updated low-rank SVD representation of the first-order momentum, closely approximating its full-rank counterpart throughout training. This factorization enables a memory-efficient fine-tuning method that adaptively updates the optimization subspace at each iteration. Crucially, MoFaSGD leverages the computed low-rank momentum factors to perform efficient spectrally normalized updates, offering an alternative to subspace moment accumulation. We establish theoretical convergence guarantees for MoFaSGD, proving it achieves an optimal rate for non-convex stochastic optimization under standard assumptions. Empirically, we demonstrate MoFaSGD's effectiveness on large language model alignment benchmarks, achieving a competitive trade-off between memory reduction (comparable to LoRA) and performance compared to state-of-the-art low-rank optimization methods. Our implementation is available at https://github.com/pmahdavi/MoFaSGD.
toXiv_bot_toot

This https://arxiv.org/abs/2505.13229 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSE_…

PARF: An Adaptive Abstraction-Strategy Tuner for Static Analysis
We launch Parf - a toolkit for adaptively tuning abstraction strategies of static program analyzers in a fully automated manner. Parf models various types of external parameters (encoding abstraction strategies) as random variables subject to probability distributions over latticed parameter spaces. It incrementally refines the probability distributions based on accumulated intermediate results generated by repeatedly sampling and analyzing, thereby ultimately yielding a set of highly accurate …

Cosmography of Non-Interacting Ghost and Generalized Ghost Dark Energy Models in f(Q) Gravity
M. Sharif, Madiha Ajmal
https://arxiv.org/abs/2507.02280 http…

Cosmography of Non-Interacting Ghost and Generalized Ghost Dark Energy Models in f(Q) Gravity
This paper aims to develop non-interacting ghost dark energy and generalized ghost dark energy models within the framework of $f(Q)$ theory using the correspondence scheme. We use pressureless matter and a power-law scale factor. The cosmic implications of the resulting models are studied through the equation of state parameter and the phase planes. We also check the stability of the reconstructed models through the squared speed of sound parameter. The equation of state parameter exhibits a ph…

«En cause, une accumulation de travaux qui congestionne toute la ville.» Ah, bien sûr, les voitures ne sont jamais le problème.
https://www.rts.ch/info/regions/geneve/2025/article/geneve-p…

This https://arxiv.org/abs/2404.11444 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_qu…

Fidelity decay and error accumulation in random quantum circuits
We present a comprehensive analysis of fidelity decay and error accumulation in faulty quantum circuit models. Our work devises an analytical bound for the average fidelity between desired and faulty output states, accounting for errors that may arise during the implementation of two-qubit gates and multi-qubit permutations. It is shown that fidelity decays exponentially with both circuit depth and the number of qubits raised to an architecture-dependent power, and determine the decay rates as …

Corrector Sampling in Language Models
Itai Gat, Neta Shaul, Uriel Singer, Yaron Lipman
https://arxiv.org/abs/2506.06215 https://arxiv…

Corrector Sampling in Language Models
Autoregressive language models accumulate errors due to their fixed, irrevocable left-to-right token generation. To address this, we propose a new sampling method called Resample-Previous-Tokens (RPT). RPT mitigates error accumulation by iteratively revisiting and potentially replacing tokens in a window of previously generated text. This method can be integrated into existing autoregressive models, preserving their next-token-prediction quality and speed. Fine-tuning a pretrained 8B parameter …

Do conditional cash transfers in childhood increase economic resilience in adulthood? Evidence from the COVID-19 pandemic shock in Ecuador
Jos\'e-Ignacio Ant\'on, Ruthy Intriago, Juan Ponce
https://arxiv.org/abs/2506.06903

Do conditional cash transfers in childhood increase economic resilience in adulthood? Evidence from the COVID-19 pandemic shock in Ecuador
The primary goal of conditional cash transfers (CCTs) is to alleviate short-term poverty while preventing the intergenerational transmission of deprivation by promoting the accumulation of human capital among children. Although a substantial body of research has evaluated the short-run impacts of CCTs, studies on their long-term effects are relatively scarce, and evidence regarding their influence on resilience to future economic shocks is limited. As human capital accumulation is expected to e…

A Prototype Hybrid Mode Cavity for Heterodyne Axion Detection
Zenghai Li, Kevin Zhou, Marco Oriunno, Asher Berlin, Sergio Calatroni, Raffaele Tito D'Agnolo, Sebastian A. R. Ellis, Philip Schuster, Sami G. Tantawi, Natalia Toro
https://arxiv.org/abs/2507.07173

A Prototype Hybrid Mode Cavity for Heterodyne Axion Detection
In the heterodyne approach to axion detection, axion dark matter induces transitions between two modes of a microwave cavity, resulting in a parametrically enhanced signal power. We describe the fabrication and characterization of a prototype normal conducting cavity specifically optimized for heterodyne detection. Corrugations on the cavity walls support linearly polarized hybrid modes which maximize the signal power while strongly suppressing noise. We demonstrate tuning mechanisms which allo…

The number and breadth of extremely mainstream businesses who aren't afraid to participate in Rhode Island PrideFest is wonderful, given the way companies have been targeted just for acknowledging basic human rights. Nice to live here.

This https://arxiv.org/abs/2505.23868 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

Noise-Robustness Through Noise: Asymmetric LoRA Adaption with Poisoning Expert
Current parameter-efficient fine-tuning methods for adapting pre-trained language models to downstream tasks are susceptible to interference from noisy data. Conventional noise-handling approaches either rely on laborious data pre-processing or employ model architecture modifications prone to error accumulation. In contrast to existing noise-process paradigms, we propose a noise-robust adaptation method via asymmetric LoRA poisoning experts (LoPE), a novel framework that enhances model robustne…