Tootfinder

On function-on-function linear quantile regression
Muge Mutis, Ufuk Beyaztas, Filiz Karaman, Han Lin Shang
https://arxiv.org/abs/2510.10792 https://arxiv.o…

On function-on-function linear quantile regression
We present two innovative functional partial quantile regression algorithms designed to accurately and efficiently estimate the regression coefficient function within the function-on-function linear quantile regression model. Our algorithms utilize functional partial quantile regression decomposition to effectively project the infinite-dimensional response and predictor variables onto a finite-dimensional space. Within this framework, the partial quantile regression components are approximated …

Information-Computation Tradeoffs for Noiseless Linear Regression with Oblivious Contamination
Ilias Diakonikolas, Chao Gao, Daniel M. Kane, John Lafferty, Ankit Pensia
https://arxiv.org/abs/2510.10665

Information-Computation Tradeoffs for Noiseless Linear Regression with Oblivious Contamination
We study the task of noiseless linear regression under Gaussian covariates in the presence of additive oblivious contamination. Specifically, we are given i.i.d.\ samples from a distribution $(x, y)$ on $\mathbb{R}^d \times \mathbb{R}$ with $x \sim \mathcal{N}(0,\mathbf{I}_d)$ and $y = x^\top β+ z$, where $z$ is drawn independently of $x$ from an unknown distribution $E$. Moreover, $z$ satisfies $\mathbb{P}_E[z = 0] = α>0$. The goal is to accurately recover the regressor $β$ to small $\ell_2…

Robust Functional Logistic Regression
Berkay Akturk, Ufuk Beyaztas, Han Lin Shang
https://arxiv.org/abs/2510.12048 https://arxiv.org/pdf/2510.12048

Robust Functional Logistic Regression
Functional logistic regression is a popular model to capture a linear relationship between binary response and functional predictor variables. However, many methods used for parameter estimation in functional logistic regression are sensitive to outliers, which may lead to inaccurate parameter estimates and inferior classification accuracy. We propose a robust estimation procedure for functional logistic regression, in which the observations of the functional predictor are projected onto a set …

Locally Linear Convergence for Nonsmooth Convex Optimization via Coupled Smoothing and Momentum
Reza Rahimi Baghbadorani, Sergio Grammatico, Peyman Mohajerin Esfahani
https://arxiv.org/abs/2511.10239 https://arxiv.org/pdf/2511.10239 https://arxiv.org/html/2511.10239
arXiv:2511.10239v1 Announce Type: new
Abstract: We propose an adaptive accelerated smoothing technique for a nonsmooth convex optimization problem where the smoothing update rule is coupled with the momentum parameter. We also extend the setting to the case where the objective function is the sum of two nonsmooth functions. With regard to convergence rate, we provide the global (optimal) sublinear convergence guarantees of O(1/k), which is known to be provably optimal for the studied class of functions, along with a local linear rate if the nonsmooth term fulfills a so-call locally strong convexity condition. We validate the performance of our algorithm on several problem classes, including regression with the l1-norm (the Lasso problem), sparse semidefinite programming (the MaxCut problem), Nuclear norm minimization with application in model free fault diagnosis, and l_1-regularized model predictive control to showcase the benefits of the coupling. An interesting observation is that although our global convergence result guarantees O(1/k) convergence, we consistently observe a practical transient convergence rate of O(1/k^2), followed by asymptotic linear convergence as anticipated by the theoretical result. This two-phase behavior can also be explained in view of the proposed smoothing rule.
toXiv_bot_toot

Crosslisted article(s) found for physics.geo-ph. https://arxiv.org/list/physics.geo-ph/new
[1/1]:
- Rethinking deep learning: linear regression remains a key benchmark in predicting terrestrial wat...
Nie, Kumar, Chen, Zhao, Skulovich, Yoo, Pflug, Ahmad, Konapala

Accelerating Regression Tasks with Quantum Algorithms
Chenghua Liu, Zhengfeng Ji
https://arxiv.org/abs/2509.24757 https://arxiv.org/pdf/2509.24757

Accelerating Regression Tasks with Quantum Algorithms
Regression is a cornerstone of statistics and machine learning, with applications spanning science, engineering, and economics. While quantum algorithms for regression have attracted considerable attention, most existing work has focused on linear regression, leaving many more complex yet practically important variants unexplored. In this work, we present a unified quantum framework for accelerating a broad class of regression tasks -- including linear and multiple regression, Lasso, Ridge, Hub…

dHPR: A Distributed Halpern Peaceman--Rachford Method for Non-smooth Distributed Optimization Problems
Zhangcheng Feng, Defeng Sun, Yancheng Yuan, Guojun Zhang
https://arxiv.org/abs/2511.10069 https://arxiv.org/pdf/2511.10069 https://arxiv.org/html/2511.10069
arXiv:2511.10069v1 Announce Type: new
Abstract: This paper introduces the distributed Halpern Peaceman--Rachford (dHPR) method, an efficient algorithm for solving distributed convex composite optimization problems with non-smooth objectives, which achieves a non-ergodic $O(1/k)$ iteration complexity regarding Karush--Kuhn--Tucker residual. By leveraging the symmetric Gauss--Seidel decomposition, the dHPR effectively decouples the linear operators in the objective functions and consensus constraints while maintaining parallelizability and avoiding additional large proximal terms, leading to a decentralized implementation with provably fast convergence. The superior performance of dHPR is demonstrated through comprehensive numerical experiments on distributed LASSO, group LASSO, and $L_1$-regularized logistic regression problems.
toXiv_bot_toot

Closed-form $\ell_r$ norm scaling with data for overparameterized linear regression and diagonal linear networks under $\ell_p$ bias
Shuofeng Zhang, Ard Louis
https://arxiv.org/abs/2509.21181

Closed-form $\ell_r$ norm scaling with data for overparameterized linear regression and diagonal linear networks under $\ell_p$ bias
For overparameterized linear regression with isotropic Gaussian design and minimum-$\ell_p$ interpolator $p\in(1,2]$, we give a unified, high-probability characterization for the scaling of the family of parameter norms $ \\{ \lVert \widehat{w_p} \rVert_r \\}_{r \in [1,p]} $ with sample size. We solve this basic, but unresolved question through a simple dual-ray analysis, which reveals a competition between a signal *spike* and a *bulk* of null coordinates in $X^\top Y$, yielding closed-form …

Cellular Learning: Scattered Data Regression in High Dimensions via Voronoi Cells
Shankar Prasad Sastry
https://arxiv.org/abs/2510.03810 https://arxiv.org/…

Cellular Learning: Scattered Data Regression in High Dimensions via Voronoi Cells
I present a regression algorithm that provides a continuous, piecewise-smooth function approximating scattered data. It is based on composing and blending linear functions over Voronoi cells, and it scales to high dimensions. The algorithm infers Voronoi cells from seed vertices and constructs a linear function for the input data in and around each cell. As the algorithm does not explicitly compute the Voronoi diagram, it avoids the curse of dimensionality. An accuracy of around 98.2% on the MN…

Risk Comparisons in Linear Regression: Implicit Regularization Dominates Explicit Regularization
Jingfeng Wu, Peter L. Bartlett, Jason D. Lee, Sham M. Kakade, Bin Yu
https://arxiv.org/abs/2509.17251

Risk Comparisons in Linear Regression: Implicit Regularization Dominates Explicit Regularization
Existing theory suggests that for linear regression problems categorized by capacity and source conditions, gradient descent (GD) is always minimax optimal, while both ridge regression and online stochastic gradient descent (SGD) are polynomially suboptimal for certain categories of such problems. Moving beyond minimax theory, this work provides instance-wise comparisons of the finite-sample risks for these algorithms on any well-specified linear regression problem. Our analysis yields three …

Learning Linear Regression with Low-Rank Tasks in-Context
Kaito Takanami, Takashi Takahashi, Yoshiyuki Kabashima
https://arxiv.org/abs/2510.04548 https://a…

Learning Linear Regression with Low-Rank Tasks in-Context
In-context learning (ICL) is a key building block of modern large language models, yet its theoretical mechanisms remain poorly understood. It is particularly mysterious how ICL operates in real-world applications where tasks have a common structure. In this work, we address this problem by analyzing a linear attention model trained on low-rank regression tasks. Within this setting, we precisely characterize the distribution of predictions and the generalization error in the high-dimensional li…

An efficient algorithm for kernel quantile regression
Shengxiang Deng, Xudong Li, Yangjing Zhang
https://arxiv.org/abs/2510.07929 https://arxiv.org/pdf/251…

An efficient algorithm for kernel quantile regression
Kernel Quantile Regression (KQR) extends classical quantile regression to nonlinear settings using kernel methods, offering powerful tools for modeling conditional distributions. However, its application to large-scale datasets is severely limited by the computational burden of the large, dense kernel matrix and the need to efficiently solve large, often ill-conditioned linear systems. Existing state-of-the-art solvers usually struggle with scalability. In this paper, we propose a novel and hig…

Bayesian Profile Regression with Linear Mixed Models (Profile-LMM) applied to Longitudinal Exposome Data
Matteo Amestoy, Mark van de Wiel, Jeroen Lakerveld, Wessel van Wieringen
https://arxiv.org/abs/2510.08304

Bayesian Profile Regression with Linear Mixed Models (Profile-LMM) applied to Longitudinal Exposome Data
Exposure to diverse non-genetic factors, known as the exposome, is a critical determinant of health outcomes. However, analyzing the exposome presents significant methodological challenges, including: high collinearity among exposures, the longitudinal nature of repeated measurements, and potential complex interactions with individual characteristics. In this paper, we address these challenges by proposing a novel statistical framework that extends Bayesian profile regression. Our method integr…

Generalized Nonnegative Structured Kruskal Tensor Regression
Xinjue Wang, Esa Ollila, Sergiy A. Vorobyov, Ammar Mian
https://arxiv.org/abs/2509.19900 https://

Generalized Nonnegative Structured Kruskal Tensor Regression
This paper introduces Generalized Nonnegative Structured Kruskal Tensor Regression (NS-KTR), a novel tensor regression framework that enhances interpretability and performance through mode-specific hybrid regularization and nonnegativity constraints. Our approach accommodates both linear and logistic regression formulations for diverse response variables while addressing the structural heterogeneity inherent in multidimensional tensor data. We integrate fused LASSO, total variation, and ridge r…

Bayesian Transfer Learning for High-Dimensional Linear Regression via Adaptive Shrinkage
Parsa Jamshidian, Donatello Telesca
https://arxiv.org/abs/2510.03449 https://

Bayesian Transfer Learning for High-Dimensional Linear Regression via Adaptive Shrinkage
We introduce BLAST, Bayesian Linear regression with Adaptive Shrinkage for Transfer, a Bayesian multi-source transfer learning framework for high-dimensional linear regression. The proposed analytical framework leverages global-local shrinkage priors together with Bayesian source selection to balance information sharing and regularization. We show how Bayesian source selection allows for the extraction of the most useful data sources, while discounting biasing information that may lead to negat…

Theory of Scaling Laws for In-Context Regression: Depth, Width, Context and Time
Blake Bordelon, Mary I. Letey, Cengiz Pehlevan
https://arxiv.org/abs/2510.01098 https://

Theory of Scaling Laws for In-Context Regression: Depth, Width, Context and Time
We study in-context learning (ICL) of linear regression in a deep linear self-attention model, characterizing how performance depends on various computational and statistical resources (width, depth, number of training steps, batch size and data per context). In a joint limit where data dimension, context length, and residual stream width scale proportionally, we analyze the limiting asymptotics for three ICL settings: (1) isotropic covariates and tasks (ISO), (2) fixed and structured covarianc…

Optimal estimation for regression discontinuity design with binary outcomes
Takuya Ishihara, Masayuki Sawada, Kohei Yata
https://arxiv.org/abs/2509.18857 https://

Uncertainty in Machine Learning
Hans Weytjens, Wouter Verbeke
https://arxiv.org/abs/2510.06007 https://arxiv.org/pdf/2510.06007

Uncertainty in Machine Learning
This book chapter introduces the principles and practical applications of uncertainty quantification in machine learning. It explains how to identify and distinguish between different types of uncertainty and presents methods for quantifying uncertainty in predictive models, including linear regression, random forests, and neural networks. The chapter also covers conformal prediction as a framework for generating predictions with predefined confidence intervals. Finally, it explores how uncerta…

Optimality and computational barriers in variable selection under dependence
Ming Gao, Bryon Aragam
https://arxiv.org/abs/2510.03990 https://arxiv.org/pdf/…

Optimality and computational barriers in variable selection under dependence
We study the optimal sample complexity of variable selection in linear regression under general design covariance, and show that subset selection is optimal while under standard complexity assumptions, efficient algorithms for this problem do not exist. Specifically, we analyze the variable selection problem and provide the optimal sample complexity with exact dependence on the problem parameters for both known and unknown sparsity settings. Moreover, we establish a sample complexity lower boun…

Use multi level models with {parsnip}: http://multilevelmod.tidymodels.org/ #rstats #ML

A mesh-free, derivative-free, matrix-free, and highly parallel localized stochastic method for high-dimensional semilinear parabolic PDEs
Shuixin Fang, Changtao Sheng, Bihao Su, Tao Zhou
https://arxiv.org/abs/2510.02635

A mesh-free, derivative-free, matrix-free, and highly parallel localized stochastic method for high-dimensional semilinear parabolic PDEs
We develop a mesh-free, derivative-free, matrix-free, and highly parallel localized stochastic method for high-dimensional semilinear parabolic PDEs. The efficiency of the proposed method is built upon four essential components: (i) a martingale formulation of the forward backward stochastic differential equation (FBSDE); (ii) a small scale stochastic particle method for local linear regression (LLR); (iii) a decoupling strategy with a matrix-free solver for the weighted least-squares system us…

Linear Regression under Missing or Corrupted Coordinates
Ilias Diakonikolas, Jelena Diakonikolas, Daniel M. Kane, Jasper C. H. Lee, Thanasis Pittas
https://arxiv.org/abs/2509.19242

Crop Spirals: Re-thinking the field layout for future robotic agriculture
Lakshan Lavan, Lanojithan Thiyagarasa, Udara Muthugala, Rajitha de Silva
https://arxiv.org/abs/2509.25091

Crop Spirals: Re-thinking the field layout for future robotic agriculture
Conventional linear crop layouts, optimised for tractors, hinder robotic navigation with tight turns, long travel distances, and perceptual aliasing. We propose a robot-centric square spiral layout with a central tramline, enabling simpler motion and more efficient coverage. To exploit this geometry, we develop a navigation stack combining DH-ResNet18 waypoint regression, pixel-to-odometry mapping, A* planning, and model predictive control (MPC). In simulations, the spiral layout yields up to 2…

Fitting sparse high-dimensional varying-coefficient models with Bayesian regression tree ensembles
Soham Ghosh, Saloni Bhogale, Sameer K. Deshpande
https://arxiv.org/abs/2510.08204

Fitting sparse high-dimensional varying-coefficient models with Bayesian regression tree ensembles
By allowing the effects of $p$ covariates in a linear regression model to vary as functions of $R$ additional effect modifiers, varying-coefficient models (VCMs) strike a compelling balance between interpretable-but-rigid parametric models popular in classical statistics and flexible-but-opaque methods popular in machine learning. But in high-dimensional settings where $p$ and/or $R$ exceed the number of observations, existing approaches to fitting VCMs fail to identify which covariates have a …

High-dimensional Analysis of Synthetic Data Selection
Parham Rezaei, Filip Kovacevic, Francesco Locatello, Marco Mondelli
https://arxiv.org/abs/2510.08123 https://

High-dimensional Analysis of Synthetic Data Selection
Despite the progress in the development of generative models, their usefulness in creating synthetic data that improve prediction performance of classifiers has been put into question. Besides heuristic principles such as "synthetic data should be close to the real data distribution", it is actually not clear which specific properties affect the generalization error. Our paper addresses this question through the lens of high-dimensional regression. Theoretically, we show that, for linear models…

A Type 2 Fuzzy Set Approach for Building Linear Linguistic Regression Analysis under Multi Uncertainty
Junzo Watada, Pei-Chun Lin, Bo Wang, Jeng-Shyang Pan, Jose Guadalupe Flores Muniz
https://arxiv.org/abs/2509.10498

A Type 2 Fuzzy Set Approach for Building Linear Linguistic Regression Analysis under Multi Uncertainty
In this paper, we propose a novel heuristic algorithm for constructing a Type-2 Fuzzy Set of the Linear Linguistic Regression (T2F-LLR) model, designed to address uncertainty and vagueness in real-world decision-making. We consider a practical scenario involving a cosmetic company's promotional planning across four product categories: Basic Face Care, Face Cleaning, Cosmetics, and Body Care, aimed at both male and female consumers. Data are collected using fuzzy linguistic questionnaires from c…

Inference in pseudo-observation-based regression using (biased) covariance estimation and naive bootstrapping
Simon Mack, Morten Overgaard, Dennis Dobler
https://arxiv.org/abs/2510.06815

Inference in pseudo-observation-based regression using (biased) covariance estimation and naive bootstrapping
We demonstrate that the usual Huber-White estimator is not consistent for the limiting covariance of parameter estimates in pseudo-observation regression approaches. By confirming that a plug-in estimator can be used instead, we obtain asymptotically exact and consistent tests for general linear hypotheses in the parameters of the model. Additionally, we confirm that naive bootstrapping can not be used for covariance estimation in the pseudo-observation model either. However, it can be used for…

Risk Phase Transitions in Spiked Regression: Alignment Driven Benign and Catastrophic Overfitting
Jiping Li, Rishi Sonthalia
https://arxiv.org/abs/2510.01414 https://

Risk Phase Transitions in Spiked Regression: Alignment Driven Benign and Catastrophic Overfitting
This paper analyzes the generalization error of minimum-norm interpolating solutions in linear regression using spiked covariance data models. The paper characterizes how varying spike strengths and target-spike alignments can affect risk, especially in overparameterized settings. The study presents an exact expression for the generalization error, leading to a comprehensive classification of benign, tempered, and catastrophic overfitting regimes based on spike strength, the aspect ratio $c=d/n…

A note on the relation between one--step, outcome regression and IPW--type estimators of parameters with the mixed bias property
Andrea Rotnitzky, Ezequiel Smucler, James M. Robins
https://arxiv.org/abs/2509.22452

A note on the relation between one--step, outcome regression and IPW--type estimators of parameters with the mixed bias property
Bruns-Smith et al. (2025) established an algebraic identity between the one-step estimator and a specific outcome regression-type estimator for a class of parameters that forms a strict subset of the class introduced in Chernozhukov et al. (2022), assuming both nuisance functions are estimated as linear combinations of given features. They conjectured that this identity extends to the broader mixed bias class introduced in Rotnitzky et al. (2021). In this note, we prove their conjecture and fur…

Vulnerability Patching Across Software Products and Software Components: A Case Study of Red Hat's Product Portfolio
Jukka Ruohonen, Sani Abdullahi, Abhishek Tiwari
https://arxiv.org/abs/2509.13117

Vulnerability Patching Across Software Products and Software Components: A Case Study of Red Hat's Product Portfolio
Motivated by software maintenance and the more recent concept of security debt, the paper presents a time series analysis of vulnerability patching of Red Hat's products and components between 1999 and 2024. According to the results based on segmented regression analysis, the amounts of vulnerable products and components have not been stable; a linear trend describes many of the series well. Nor do the amounts align well with trends characterizing vulnerabilities in general. There are also visi…

Preventing Model Collapse Under Overparametrization: Optimal Mixing Ratios for Interpolation Learning and Ridge Regression
Anvit Garg, Sohom Bhattacharya, Pragya Sur
https://arxiv.org/abs/2509.22341

Preventing Model Collapse Under Overparametrization: Optimal Mixing Ratios for Interpolation Learning and Ridge Regression
Model collapse occurs when generative models degrade after repeatedly training on their own synthetic outputs. We study this effect in overparameterized linear regression in a setting where each iteration mixes fresh real labels with synthetic labels drawn from the model fitted in the previous iteration. We derive precise generalization error formulae for minimum-$\ell_2$-norm interpolation and ridge regression under this iterative scheme. Our analysis reveals intriguing properties of the optim…

Assisting the Grading of a Handwritten General Chemistry Exam with Artificial Intelligence
Jan Cvengros, Gerd Kortemeyer
https://arxiv.org/abs/2509.10591 https://

Assisting the Grading of a Handwritten General Chemistry Exam with Artificial Intelligence
We explore the effectiveness and reliability of an artificial intelligence (AI)-based grading system for a handwritten general chemistry exam, comparing AI-assigned scores to human grading across various types of questions. Exam pages and grading rubrics were uploaded as images to account for chemical reaction equations, short and long open-ended answers, numerical and symbolic answer derivations, drawing, and sketching in pencil-and-paper format. Using linear regression analyses and psychometr…

Crosslisted article(s) found for stat.CO. https://arxiv.org/list/stat.CO/new
[1/1]:
- Bayesian Transfer Learning for High-Dimensional Linear Regression via Adaptive Shrinkage
Parsa Jamshidian, Donatello Telesca

Data coarse graining can improve model performance
Alex Nguyen, David J. Schwab, Vudtiwat Ngampruetikorn
https://arxiv.org/abs/2509.14498 https://arxiv.org…

Data coarse graining can improve model performance
Lossy data transformations by definition lose information. Yet, in modern machine learning, methods like data pruning and lossy data augmentation can help improve generalization performance. We study this paradox using a solvable model of high-dimensional, ridge-regularized linear regression under 'data coarse graining.' Inspired by the renormalization group in statistical physics, we analyze coarse-graining schemes that systematically discard features based on their relevance to the learning t…

The Impact of AI Adoption on Retail Across Countries and Industries
Yunqi Liu
https://arxiv.org/abs/2509.15885 https://arxiv.org/pdf/2509.15885

The Impact of AI Adoption on Retail Across Countries and Industries
This study investigates the impact of artificial intelligence (AI) adoption on job loss rates using the Global AI Content Impact Dataset (2020--2025). The panel comprises 200 industry-country-year observations across Australia, China, France, Japan, and the United Kingdom in ten industries. A three-stage ordinary least squares (OLS) framework is applied. First, a full-sample regression finds no significant linear association between AI adoption rate and job loss rate ($β\approx -0.0026$, $p = …

Mathematical Theory of Collinearity Effects on Machine Learning Variable Importance Measures
Kelvyn K. Bladen, D. Richard Cutler, Alan Wisler
https://arxiv.org/abs/2510.00557 ht…

Mathematical Theory of Collinearity Effects on Machine Learning Variable Importance Measures
In many machine learning problems, understanding variable importance is a central concern. Two common approaches are Permute-and-Predict (PaP), which randomly permutes a feature in a validation set, and Leave-One-Covariate-Out (LOCO), which retrains models after permuting a training feature. Both methods deem a variable important if predictions with the original data substantially outperform those with permutations. In linear regression, empirical studies have linked PaP to regression coefficie…

Incorporating priors in learning: a random matrix study under a teacher-student framework
Malik Tiomoko, Ekkehard Schnoor
https://arxiv.org/abs/2509.22124 https://

Incorporating priors in learning: a random matrix study under a teacher-student framework
Regularized linear regression is central to machine learning, yet its high-dimensional behavior with informative priors remains poorly understood. We provide the first exact asymptotic characterization of training and test risks for maximum a posteriori (MAP) regression with Gaussian priors centered at a domain-informed initialization. Our framework unifies ridge regression, least squares, and prior-informed estimators, and -- using random matrix theory -- yields closed-form risk formulas that …

Assumption-lean Inference for Network-linked Data
Wei Li, Nilanjan Chakraborty, Robert Lunde
https://arxiv.org/abs/2510.00287 https://arxiv.org/pdf/2510.00…

Assumption-lean Inference for Network-linked Data
We consider statistical inference for network-linked regression problems, where covariates may include network summary statistics computed for each node. In settings involving network data, it is often natural to posit that latent variables govern connection probabilities in the graph. Since the presence of these latent features makes classical regression assumptions even less tenable, we propose an assumption-lean framework for linear regression with jointly exchangeable regression arrays. We …

On the Rate of Gaussian Approximation for Linear Regression Problems
Marat Khusainov, Marina Sheshukova, Alain Durmus, Sergey Samsonov
https://arxiv.org/abs/2509.14039 https://

On the Rate of Gaussian Approximation for Linear Regression Problems
In this paper, we consider the problem of Gaussian approximation for the online linear regression task. We derive the corresponding rates for the setting of a constant learning rate and study the explicit dependence of the convergence rate upon the problem dimension $d$ and quantities related to the design matrix. When the number of iterations $n$ is known in advance, our results yield the rate of normal approximation of order $\sqrt{\log{n}/n}$, provided that the sample size $n$ is large enoug…

Some Simplifications for the Expectation-Maximization (EM) Algorithm: The Linear Regression Model Case
Daniel A. Griffith
https://arxiv.org/abs/2509.19461 https://

Some Simplifications for the Expectation-Maximization (EM) Algorithm: The Linear Regression Model Case
The EM algorithm is a generic tool that offers maximum likelihood solutions when datasets are incomplete with data values missing at random or completely at random. At least for its simplest form, the algorithm can be rewritten in terms of an ANCOVA regression specification. This formulation allows several analytical results to be derived that permit the EM algorithm solution to be expressed in terms of new observation predictions and their variances. Implementations can be made with a linear r…

Optimal Nuisance Function Tuning for Estimating a Doubly Robust Functional under Proportional Asymptotics
Sean McGrath, Debarghya Mukherjee, Rajarshi Mukherjee, Zixiao Jolene Wang
https://arxiv.org/abs/2509.25536

Optimal Nuisance Function Tuning for Estimating a Doubly Robust Functional under Proportional Asymptotics
In this paper, we explore the asymptotically optimal tuning parameter choice in ridge regression for estimating nuisance functions of a statistical functional that has recently gained prominence in conditional independence testing and causal inference. Given a sample of size $n$, we study estimators of the Expected Conditional Covariance (ECC) between variables $Y$ and $A$ given a high-dimensional covariate $X \in \mathbb{R}^p$. Under linear regression models for $Y$ and $A$ on $X$ and the prop…

Adaptive randomized pivoting and volume sampling
Ethan N. Epperly
https://arxiv.org/abs/2510.02513 https://arxiv.org/pdf/2510.02513

Adaptive randomized pivoting and volume sampling
Adaptive randomized pivoting (ARP) is a recently proposed and highly effective algorithm for column subset selection. This paper reinterprets the ARP algorithm by drawing connections to the volume sampling distribution and active learning algorithms for linear regression. As consequences, this paper presents new analysis for the ARP algorithm and faster implementations using rejection sampling.

Pretrain-Test Task Alignment Governs Generalization in In-Context Learning
Mary I. Letey, Jacob A. Zavatone-Veth, Yue M. Lu, Cengiz Pehlevan
https://arxiv.org/abs/2509.26551 htt…

Pretrain-Test Task Alignment Governs Generalization in In-Context Learning
In-context learning (ICL) is a central capability of Transformer models, but the structures in data that enable its emergence and govern its robustness remain poorly understood. In this work, we study how the structure of pretraining tasks governs generalization in ICL. Using a solvable model for ICL of linear regression by linear attention, we derive an exact expression for ICL generalization error in high dimensions under arbitrary pretraining-testing task covariance mismatch. This leads to a…

KOO Method-based Consistent Clustering for Group-wise Linear Regression with Graph Structure
M. Ohishi, R. Oda
https://arxiv.org/abs/2509.11103 https://arx…

KOO Method-based Consistent Clustering for Group-wise Linear Regression with Graph Structure
The kick-one-out (KOO) method is a variable selection method based on a model selection criterion. The method is very simple, and yet it has consistency in variable selection under a high-dimensional asymptotic framework with a specific model selection criterion. This paper proposes the join-twotogether (JTT) method, which is a clustering method based on the KOO method for group-wise linear regression with graph structure. The JTT method formulates the clustering problem as an edge selection pr…

Fast Estimation of Wasserstein Distances via Regression on Sliced Wasserstein Distances
Khai Nguyen, Hai Nguyen, Nhat Ho
https://arxiv.org/abs/2509.20508 https://

Fast Estimation of Wasserstein Distances via Regression on Sliced Wasserstein Distances
We address the problem of efficiently computing Wasserstein distances for multiple pairs of distributions drawn from a meta-distribution. To this end, we propose a fast estimation method based on regressing Wasserstein distance on sliced Wasserstein (SW) distances. Specifically, we leverage both standard SW distances, which provide lower bounds, and lifted SW distances, which provide upper bounds, as predictors of the true Wasserstein distance. To ensure parsimony, we introduce two linear model…

Guaranteed Noisy CP Tensor Recovery via Riemannian Optimization on the Segre Manifold
Ke Xu, Yuefeng Han
https://arxiv.org/abs/2510.00569 https://arxiv.org…

Guaranteed Noisy CP Tensor Recovery via Riemannian Optimization on the Segre Manifold
Recovering a low-CP-rank tensor from noisy linear measurements is a central challenge in high-dimensional data analysis, with applications spanning tensor PCA, tensor regression, and beyond. We exploit the intrinsic geometry of rank-one tensors by casting the recovery task as an optimization problem over the Segre manifold, the smooth Riemannian manifold of rank-one tensors. This geometric viewpoint yields two powerful algorithms: Riemannian Gradient Descent (RGD) and Riemannian Gauss-Newton (R…

Repro Samples Method for Model-Free Inference in High-Dimensional Binary Classification
Xiaotian Hou, Peng Wang, Minge Xie, Linjun Zhang
https://arxiv.org/abs/2510.01468 https:/…

Repro Samples Method for Model-Free Inference in High-Dimensional Binary Classification
This paper presents a novel method for statistical inference in high-dimensional binary models with unspecified structure, where we leverage a (potentially misspecified) sparsity-constrained working generalized linear model (GLM) to facilitate the inference process. Our method is based on the repro samples framework, which generates artificial samples that mimic the actual data-generating process. Our inference targets include the model support, case probabilities, and the oracle regression coe…

Least squares-based methods to bias adjustment in scalar-on-function regression model using a functional instrumental variable
Xiwei Chen, Ufuk Beyaztas, Caihong Qin, Heyang Ji, Gilson Honvoh, Roger S. Zoh, Lan Xue, Carmen D. Tekwe
https://arxiv.org/abs/2509.12122

Least squares-based methods to bias adjustment in scalar-on-function regression model using a functional instrumental variable
Instrumental variables are widely used to adjust for measurement error bias when assessing associations of health outcomes with ME prone independent variables. IV approaches addressing ME in longitudinal models are well established, but few methods exist for functional regression. We develop two methods to adjust for ME bias in scalar on function linear models. We regress a scalar outcome on an ME prone functional variable using a functional IV for model identification and propose two least squ…

A Random Matrix Perspective of Echo State Networks: From Precise Bias--Variance Characterization to Optimal Regularization
Yessin Moakher, Malik Tiomoko, Cosme Louart, Zhenyu Liao
https://arxiv.org/abs/2509.22011

A Random Matrix Perspective of Echo State Networks: From Precise Bias--Variance Characterization to Optimal Regularization
We present a rigorous asymptotic analysis of Echo State Networks (ESNs) in a teacher student setting with a linear teacher with oracle weights. Leveraging random matrix theory, we derive closed form expressions for the asymptotic bias, variance, and mean-squared error (MSE) as functions of the input statistics, the oracle vector, and the ridge regularization parameter. The analysis reveals two key departures from classical ridge regression: (i) ESNs do not exhibit double descent, and (ii) ESNs …