Tootfinder

Online and Offline Robust Multivariate Linear Regression
Antoine Godichon-Baggioni (LPSM), Stephane S. Robin (LPSM), Laure Sansonnet (MIA Paris-Saclay, LPSM)
https://arxiv.org/abs/2404.19496

Online and Offline Robust Multivariate Linear Regression
We consider the robust estimation of the parameters of multivariate Gaussian linear regression models. To this aim we consider robust version of the usual (Mahalanobis) least-square criterion, with or without Ridge regularization. We introduce two methods each considered contrast: (i) online stochastic gradient descent algorithms and their averaged versions and (ii) offline fix-point algorithms. Under weak assumptions, we prove the asymptotic normality of the resulting estimates. Because the…

Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality
Siyu Chen, Heejune Sheen, Tianhao Wang, Zhuoran Yang
https://arxiv.org/abs/2402.19442

Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality
We study the dynamics of gradient flow for training a multi-head softmax attention model for in-context learning of multi-task linear regression. We establish the global convergence of gradient flow under suitable choices of initialization. In addition, we prove that an interesting "task allocation" phenomenon emerges during the gradient flow dynamics, where each attention head focuses on solving a single task of the multi-task model. Specifically, we prove that the gradient flow dynamics can b…

This https://arxiv.org/abs/2402.18065 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csRO_…

A Probabilistic Motion Model for Skid-Steer Wheeled Mobile Robot Navigation on Off-Road Terrains
Skid-Steer Wheeled Mobile Robots (SSWMRs) are increasingly being used for off-road autonomy applications. When turning at high speeds, these robots tend to undergo significant skidding and slipping. In this work, using Gaussian Process Regression (GPR) and Sigma-Point Transforms, we estimate the non-linear effects of tire-terrain interaction on robot velocities in a probabilistic fashion. Using the mean estimates from GPR, we propose a data-driven dynamic motion model that is more accurate at p…

Meta-Learning with Generalized Ridge Regression: High-dimensional Asymptotics, Optimality and Hyper-covariance Estimation
Yanhao Jin, Krishnakumar Balasubramanian, Debashis Paul
https://arxiv.org/abs/2403.19720

Meta-Learning with Generalized Ridge Regression: High-dimensional Asymptotics, Optimality and Hyper-covariance Estimation
Meta-learning involves training models on a variety of training tasks in a way that enables them to generalize well on new, unseen test tasks. In this work, we consider meta-learning within the framework of high-dimensional multivariate random-effects linear models and study generalized ridge-regression based predictions. The statistical intuition of using generalized ridge regression in this setting is that the covariance structure of the random regression coefficients could be leveraged to ma…

A network-constrain Weibull AFT model for biomarkers discovery
Claudia Angelini, Daniela De Canditiis, Italia De Feis, Antonella Iuliano
https://arxiv.org/abs/2402.18242

A network-constrain Weibull AFT model for biomarkers discovery
We propose AFTNet, a novel network-constraint survival analysis method based on the Weibull accelerated failure time (AFT) model solved by a penalized likelihood approach for variable selection and estimation. When using the log-linear representation, the inference problem becomes a structured sparse regression problem for which we explicitly incorporate the correlation patterns among predictors using a double penalty that promotes both sparsity and grouping effect. Moreover, we establish the t…

On Bootstrapping Lasso in Generalized Linear Models and the Cross Validation
Mayukh Choudhury, Debraj Das
https://arxiv.org/abs/2403.19515 https://<…

On Bootstrapping Lasso in Generalized Linear Models and the Cross Validation
Generalized linear models or GLM constitutes an important set of models which generalizes the ordinary linear regression by connecting the response variable with the covariates through arbitrary link functions. On the other hand, Lasso is a popular and easy to implement penalization method in regression when all the covariates are not relevant. However, Lasso generally has non-tractable asymptotic distribution and hence development of an alternative method of distributional approximation is req…

Application of Deep Learning for Factor Timing in Asset Management
Prabhu Prasad Panda, Maysam Khodayari Gharanchaei, Xilin Chen, Haoshu Lyu
https://arxiv.org/abs/2404.18017

Application of Deep Learning for Factor Timing in Asset Management
The paper examines the performance of regression models (OLS linear regression, Ridge regression, Random Forest, and Fully-connected Neural Network) on the prediction of CMA (Conservative Minus Aggressive) factor premium and the performance of factor timing investment with them. Out-of-sample R-squared shows that more flexible models have better performance in explaining the variance in factor premium of the unseen period, and the back testing affirms that the factor timing based on more flexib…

Improving the Bit Complexity of Communication for Distributed Convex Optimization
Mehrdad Ghadiri, Yin Tat Lee, Swati Padmanabhan, William Swartworth, David Woodruff, Guanghao Ye
https://arxiv.org/abs/2403.19146

Improving the Bit Complexity of Communication for Distributed Convex Optimization
We consider the communication complexity of some fundamental convex optimization problems in the point-to-point (coordinator) and blackboard communication models. We strengthen known bounds for approximately solving linear regression, $p$-norm regression (for $1\leq p\leq 2$), linear programming, minimizing the sum of finitely many convex nonsmooth functions with varying supports, and low rank approximation; for a number of these fundamental problems our bounds are nearly optimal, as proven by …

On properties of fractional posterior in generalized reduced-rank regression
The Tien Mai
https://arxiv.org/abs/2404.17850 https://ar…

On properties of fractional posterior in generalized reduced-rank regression
Reduced rank regression (RRR) is a widely employed model for investigating the linear association between multiple response variables and a set of predictors. While RRR has been extensively explored in various works, the focus has predominantly been on continuous response variables, overlooking other types of outcomes. This study shifts its attention to the Bayesian perspective of generalized linear models (GLM) within the RRR framework. In this work, we relax the requirement for the link funct…

This https://arxiv.org/abs/2302.09727 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Statistical Inference for Linear Functionals of Online SGD in High-dimensional Linear Regression
Stochastic gradient descent (SGD) has emerged as the quintessential method in a data scientist's toolbox. Using SGD for high-stakes applications requires, however, careful quantification of the associated uncertainty. Towards that end, in this work, we establish a high-dimensional Central Limit Theorem (CLT) for linear functionals of online SGD iterates for overparametrized least-squares regression with non-isotropic Gaussian inputs. Our result shows that a CLT holds even when the dimensionalit…

On Bootstrapping Lasso in Generalized Linear Models and the Cross Validation
Mayukh Choudhury, Debraj Das
https://arxiv.org/abs/2403.19515 https://<…

On Bootstrapping Lasso in Generalized Linear Models and the Cross Validation
Generalized linear models or GLM constitutes an important set of models which generalizes the ordinary linear regression by connecting the response variable with the covariates through arbitrary link functions. On the other hand, Lasso is a popular and easy to implement penalization method in regression when all the covariates are not relevant. However, Lasso generally has non-tractable asymptotic distribution and hence development of an alternative method of distributional approximation is req…

Sparse Linear Regression and Lattice Problems
Aparna Gupte, Neekon Vafa, Vinod Vaikuntanathan
https://arxiv.org/abs/2402.14645 https://

Sparse Linear Regression and Lattice Problems
Sparse linear regression (SLR) is a well-studied problem in statistics where one is given a design matrix $X\in\mathbb{R}^{m\times n}$ and a response vector $y=Xθ^*+w$ for a $k$-sparse vector $θ^*$ (that is, $\|θ^*\|_0\leq k$) and small, arbitrary noise $w$, and the goal is to find a $k$-sparse $\widehatθ \in \mathbb{R}^n$ that minimizes the mean squared prediction error $\frac{1}{m}\|X\widehatθ-Xθ^*\|^2_2$. While $\ell_1$-relaxation methods such as basis pursuit, Lass…

This https://arxiv.org/abs/2206.04277 has been replaced.
link: https://scholar.google.com/scholar?q=a

On Hypothesis Transfer Learning of Functional Linear Models
We study the transfer learning (TL) for the functional linear regression (FLR) under the Reproducing Kernel Hilbert Space (RKHS) framework, observing the TL techniques in existing high-dimensional linear regression is not compatible with the truncation-based FLR methods as functional data are intrinsically infinite-dimensional and generated by smooth underlying processes. We measure the similarity across tasks using RKHS distance, allowing the type of information being transferred tied to the p…

Die #Erderwärmung führt zu steigenden Preisen! Höhere Temperaturen lassen weltweit die Kosten für #Lebensmittel und andere Waren steigen - und das trifft alle, sowohl in reichen als auch in ärmeren Ländern. Besonders betroffen: Regionen am

Global warming and heat extremes to enhance inflationary pressures - Communications Earth & Environment
Future global warming and intensifying heat extremes will have strong, non-linear and persistent upward impacts on inflation across the world, according to an analysis that combines physical projections with causal historical impacts determined by fixed-effects regression analysis.

Essential Properties of Type III* Methods
Lynn Roy LaMotte
https://arxiv.org/abs/2402.19029 https://arxiv.org/pdf/2402.19029

Essential Properties of Type III* Methods
Type III methods, introduced by SAS in 1976, formulate estimable functions that substitute, somehow, for classical ANOVA effects in multiple linear regression models. They have been controversial since, provoking wide use and satisfied users on the one hand and skepticism and scorn on the other. Their essential mathematical properties have not been established, although they are widely thought to be known: what those functions are, to what extent they coincide with classical ANOVA effects, and …

On Machine Learning Complete Intersection Calabi-Yau 3-folds
Kaniba Mady Keita
https://arxiv.org/abs/2404.11710 https://arxiv.org/pdf…

On Machine Learning Complete Intersection Calabi-Yau 3-folds
Gaussian Process Regression, Kernel Support Vector Regression, the random forest, extreme gradient boosting and the generalized linear model algorithms are applied to data of Complete Intersection Calabi-Yau 3-folds. It is shown that Gaussian process regression is the most suitable for learning the Hodge number h^(2,1)in terms of h^(1,1). The performance of this regression algorithm is such that the Pearson correlation coefficient for the validation set is R^2 = 0.9999999995 with a Root Mean Sq…

This https://arxiv.org/abs/2401.11229 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_eco…

Estimation with Pairwise Observations
The paper introduces a new estimation method for the standard linear regression model. The procedure is not driven by the optimisation of any objective function rather, it is a simple weighted average of slopes from observation pairs. The paper shows that such estimator is consistent for carefully selected weights. Other properties, such as asymptotic distributions, have also been derived to facilitate valid statistical inference. Unlike traditional methods, such as Least Squares and Maximum Li…

This https://arxiv.org/abs/2312.05382 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Estimation Sample Complexity of a Class of Nonlinear Continuous-time Systems
We present a method of parameter estimation for large class of nonlinear systems, namely those in which the state consists of output derivatives and the flow is linear in the parameter. The method, which solves for the unknown parameter by directly inverting the dynamics using regularized linear regression, is based on new design and analysis ideas for differentiation filtering and regularized least squares. Combined in series, they yield a novel finite-sample bound on mean absolute error of es…

Use multi level models with {parsnip}: http://multilevelmod.tidymodels.org/ #rstats #ML

Order Estimation of Linear-Phase FIR Filters for DAC Equalization in Multiple Nyquist Bands
Deijany Rodriguez Linares, H{\aa}kan Johansson, Yinan Wang
https://arxiv.org/abs/2402.12075

Order Estimation of Linear-Phase FIR Filters for DAC Equalization in Multiple Nyquist Bands
This letter considers the design of linear-phase finite-length impulse response (FIR) filters for equalization of the frequency responses of digital-to-analog converters (DACs). The letter derives estimates for the filter orders required, as functions of the bandwidth and equalization accuracy, for four DAC pulses that are used in DACs in multiple Nyquist bands. The estimates are derived through a large set of minimax-optimal equalizers and the use of symbolic regression followed by minimax-opt…

This https://arxiv.org/abs/2303.16599 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Difference-based covariance matrix estimate in time series nonparametric regression with applications to specification tests
Long-run covariance matrix estimation is the building block of time series inference. The corresponding difference-based estimator, which avoids detrending, has attracted considerable interest due to its robustness to both smooth and abrupt structural breaks and its competitive finite sample performance. However, existing methods mainly focus on estimators for the univariate process while their direct and multivariate extensions for most linear models are asymptotically biased. We propose a nov…

This https://arxiv.org/abs/2402.15213 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Statistical Agnostic Regression: a machine learning method to validate regression models
Regression analysis is a central topic in statistical modeling, aiming to estimate the relationships between a dependent variable, commonly referred to as the response variable, and one or more independent variables, i.e., explanatory variables. Linear regression is by far the most popular method for performing this task in several fields of research, such as prediction, forecasting, or causal inference. Beyond various classical methods to solve linear regression problems, such as Ordinary Leas…

This https://arxiv.org/abs/2310.04918 has been replaced.
link: https://scholar.google.com/scholar?q=a

SWAP: Sparse Entropic Wasserstein Regression for Robust Network Pruning
This study tackles the issue of neural network pruning that inaccurate gradients exist when computing the empirical Fisher Information Matrix (FIM). We introduce SWAP, an Entropic Wasserstein regression (EWR) network pruning formulation, capitalizing on the geometric attributes of the optimal transport (OT) problem. The "swap" of a commonly used standard linear regression (LR) with the EWR in optimization is analytically showcased to excel in noise mitigation by adopting neighborhood interpolat…

Estimation of non-uniform blur using a patch-based regression convolutional neural network (CNN)
Luis G. Varela, Laura E. Boucheron, Steven Sandoval, David Voelz, Abu Bucker Siddik
https://arxiv.org/abs/2402.07796

Estimation of non-uniform blur using a patch-based regression convolutional neural network (CNN)
The non-uniform blur of atmospheric turbulence can be modeled as a superposition of linear motion blur kernels at a patch level. We propose a regression convolutional neural network (CNN) to predict angle and length of a linear motion blur kernel for varying sized patches. We analyze the robustness of the network for different patch sizes and the performance of the network in regions where the characteristics of the blur are transitioning. Alternating patch sizes per epoch in training, we find …

This https://arxiv.org/abs/2306.10405 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

A semi-parametric estimation method for quantile coherence with an application to bivariate financial time series clustering
In multivariate time series analysis, spectral coherence measures the linear dependency between two time series at different frequencies. However, real data applications often exhibit nonlinear dependency in the frequency domain. Conventional coherence analysis fails to capture such dependency. The quantile coherence, on the other hand, characterizes nonlinear dependency by defining the coherence at a set of quantile levels based on trigonometric quantile regression. This paper introduces a new…

One-Bit Quantization and Sparsification for Multiclass Linear Classification via Regularized Regression
Reza Ghane, Danil Akhtiamov, Babak Hassibi
https://arxiv.org/abs/2402.10474

One-Bit Quantization and Sparsification for Multiclass Linear Classification via Regularized Regression
We study the use of linear regression for multiclass classification in the over-parametrized regime where some of the training data is mislabeled. In such scenarios it is necessary to add an explicit regularization term, $λf(w)$, for some convex function $f(\cdot)$, to avoid overfitting the mislabeled data. In our analysis, we assume that the data is sampled from a Gaussian Mixture Model with equal class sizes, and that a proportion $c$ of the training labels is corrupted for each class. Under…

This https://arxiv.org/abs/2402.15213 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Statistical Agnostic Regression: a machine learning method to validate regression models
Regression analysis is a central topic in statistical modeling, aiming to estimate the relationships between a dependent variable, commonly referred to as the response variable, and one or more independent variables, i.e., explanatory variables. Linear regression is by far the most popular method for performing this task in several fields of research, such as prediction, forecasting, or causal inference. Beyond various classical methods to solve linear regression problems, such as Ordinary Leas…

This https://arxiv.org/abs/2207.13289 has been replaced.
link: https://scholar.google.com/scholar?q=a

Analyzing the Differentially Private Theil-Sen Estimator for Simple Linear Regression
In this paper, we study differentially private point and confidence interval estimators for simple linear regression. Motivated by recent work that highlights the strong empirical performance of an algorithm based on robust statistics, DPTheilSen, we provide a rigorous, finite-sample analysis of its privacy and accuracy properties, offer guidance on setting hyperparameters, and show how to produce differentially private confidence intervals to accompany its point estimates.

This https://arxiv.org/abs/2309.14596 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Model averaging: A shrinkage perspective
Model averaging (MA), a technique for combining estimators from a set of candidate models, has attracted increasing attention in machine learning and statistics. In the existing literature, there is an implicit understanding that MA can be viewed as a form of shrinkage estimation that draws the response vector towards the subspaces spanned by the candidate models. This paper explores this perspective by establishing connections between MA and shrinkage in a linear regression setting with multip…

This https://arxiv.org/abs/2112.02712 has been replaced.
link: https://scholar.google.com/scholar?q=a

Interpretable discriminant analysis for functional data supported on random nonlinear domains with an application to Alzheimer's disease
We introduce a novel framework for the classification of functional data supported on nonlinear, and possibly random, manifold domains. The motivating application is the identification of subjects with Alzheimer's disease from their cortical surface geometry and associated cortical thickness map. The proposed model is based upon a reformulation of the classification problem as a regularized multivariate functional linear regression model. This allows us to adopt a direct approach to the estimat…

This https://arxiv.org/abs/2306.03816 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Parametrization, Prior Independence, and the Semiparametric Bernstein-von Mises Theorem for the Partially Linear Model
I prove a semiparametric Bernstein-von Mises theorem for a partially linear regression model with independent priors for the low-dimensional parameter of interest and the infinite-dimensional nuisance parameters. My result avoids a prior invariance condition that arises from a loss of information in not knowing the nuisance parameter. The key idea is a feasible reparametrization of the regression function that mimics the Gaussian profile likelihood. This allows a researcher to assume independen…

Tailors: New Music Timbre Visualizer to Entertain Music Through Imagery
ChungHa Lee
https://arxiv.org/abs/2404.15181 https://arxiv.or…

Tailors: New Music Timbre Visualizer to Entertain Music Through Imagery
In this paper, I have implemented a timbre visualization system called Tailors. Through the experiment with 27 MIR users, Tailors was found to be effective in conveying timbral warmth, brightness, depth, shallowness, hardness, roughness, and sharpness features of music compared to the only music condition and basic visualization. All scores of Tailors in the music imagery and music entertainment surveys were valued highest among the three conditions. Multiple linear regression analysis between …

Regression Discontinuity Design with Spillovers
Eric Auerbach, Yong Cai, Ahnaf Rafi
https://arxiv.org/abs/2404.06471 https://arxiv.org/pdf/2404.06471
arXiv:2404.06471v1 Announce Type: new
Abstract: Researchers who estimate treatment effects using a regression discontinuity design (RDD) typically assume that there are no spillovers between the treated and control units. This may be unrealistic. We characterize the estimand of RDD in a setting where spillovers occur between units that are close in their values of the running variable. Under the assumption that spillovers are linear-in-means, we show that the estimand depends on the ratio of two terms: (1) the radius over which spillovers occur and (2) the choice of bandwidth used for the local linear regression. Specifically, RDD estimates direct treatment effect when radius is of larger order than the bandwidth, and total treatment effect when radius is of smaller order than the bandwidth. In the more realistic regime where radius is of similar order as the bandwidth, the RDD estimand is a mix of the above effects. To recover direct and spillover effects, we propose incorporating estimated spillover terms into local linear regression -- the local analog of peer effects regression. We also clarify the settings under which the donut-hole RD is able to eliminate the effects of spillovers.

The Rule of link functions on Binomial Regression Model: A Cross Sectional Study on Child Malnutrition, Bangladesh
Md Mehedi Hasan Bhuiyan
https://arxiv.org/abs/2403.17948

The Rule of link functions on Binomial Regression Model: A Cross Sectional Study on Child Malnutrition, Bangladesh
Link function is a key tool in the binomial regression model defined as non-linear model under GLM approach. It transforms the nonlinear regression to linear model with converting the interval (-\infty,\infty) to the probability [0,1]. The binomial model with link functions (logit, probit, cloglog and cauchy) are applied on the proportional of child malnutrition age 0-5 years in each household level. Multiple Indicator Cluster survey (MICS)-2019, Bangladesh was conducted by a joint cooperation …

Distributed and Rate-Adaptive Feature Compression
Aditya Deshmukh, Venugopal V. Veeravalli, Gunjan Verma
https://arxiv.org/abs/2404.02179 https://

Distributed and Rate-Adaptive Feature Compression
We study the problem of distributed and rate-adaptive feature compression for linear regression. A set of distributed sensors collect disjoint features of regressor data. A fusion center is assumed to contain a pretrained linear regression model, trained on a dataset of the entire uncompressed data. At inference time, the sensors compress their observations and send them to the fusion center through communication-constrained channels, whose rates can change with time. Our goal is to design a fe…

Linear Discriminant Regularized Regression
Xin Bing, Bingqing Li, Marten Wegkamp
https://arxiv.org/abs/2402.14260 https://arxiv.org/p…

Linear Discriminant Regularized Regression
Linear Discriminant Analysis (LDA) is an important classification approach. Its simple linear form makes it easy to interpret and it is capable to handle multi-class responses. It is closely related to other classical multivariate statistical techniques, such as Fisher's discriminant analysis, canonical correlation analysis and linear regression. In this paper we strengthen its connection to multivariate response regression by characterizing the explicit relationship between the discriminant di…

Determinants of Uruguay's Real Effective Exchange Rate: A Mundell-Fleming Model Approach
Didarul Islam, Mohammad Abdullah Al Faisal
https://arxiv.org/abs/2403.16452

Determinants of Uruguay's Real Effective Exchange Rate: A Mundell-Fleming Model Approach
This study examines the factors influencing the short-term real effective exchange rate (REER) in Uruguay by applying an extended Mundell-Fleming model. Analyzing the impact of the US lending rate (USLR), money supply (M2), inflation (CPI), and the world interest rate (WIR), the paper uses a linear regression model with Newey-West standard errors. Key findings reveal that an increase in the USLR, CPI, and M2 is associated with a depreciation of the REER. In contrast, WIR shows no significant im…

This https://arxiv.org/abs/2308.04921 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

How to induce regularization in linear models: A guide to reparametrizing gradient flow
In this work, we analyze the relation between reparametrizations of gradient flow and the induced implicit bias in linear models, which encompass various basic regression tasks. In particular, we aim at understanding the influence of the model parameters - reparametrization, loss, and link function - on the convergence behavior of gradient flow. Our results provide conditions under which the implicit bias can be well-described and convergence of the flow is guaranteed. We furthermore show how t…

Strategies for Machine Learning Applied to Noisy HEP Datasets: Modular Solid State Detectors from SuperCDMS
P. B. Cushman, M. C. Fritts, A. D. Chambers, A. Roy, T. Li
https://arxiv.org/abs/2404.10971

Strategies for Machine Learning Applied to Noisy HEP Datasets: Modular Solid State Detectors from SuperCDMS
Background reduction in the SuperCDMS dark matter experiment depends on removing surface events within individual detectors by identifying the location of each incident particle interaction. Position reconstruction is achieved by combining pulse shape information over multiple phonon channels, a task well-suited to machine learning techniques. Data from an Am-241 scan of a SuperCDMS SNOLAB detector was used to study a selection of statistical approaches, including linear regression, artificial …

This https://arxiv.org/abs/2308.01381 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Estimation of motion blur kernel parameters using regression convolutional neural networks
Many deblurring and blur kernel estimation methods use MAP or classification deep learning techniques to sharpen an image and predict the blur kernel. We propose a regression approach using neural networks to predict the parameters of linear motion blur kernels. These kernels can be parameterized by its length of blur and the orientation of the blur.This paper will analyze the relationship between length and angle of linear motion blur. This analysis will help establish a foundation to using re…

This https://arxiv.org/abs/2403.17481 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

A Type of Nonlinear Fréchet Regressions
The existing Fréchet regression is actually defined within a linear framework, since the weight function in the Fréchet objective function is linearly defined, and the resulting Fréchet regression function is identified to be a linear model when the random object belongs to a Hilbert space. Even for nonparametric and semiparametric Fréchet regressions, which are usually nonlinear, the existing methods handle them by local linear (or local polynomial) technique, and the resulting Fréchet re…

skscope: Fast Sparsity-Constrained Optimization in Python
Zezhi Wang, Jin Zhu, Peng Chen, Huiyang Peng, Xiaoke Zhang, Anran Wang, Yu Zheng, Junxian Zhu, Xueqin Wang
https://arxiv.org/abs/2403.18540

skscope: Fast Sparsity-Constrained Optimization in Python
Applying iterative solvers on sparsity-constrained optimization (SCO) requires tedious mathematical deduction and careful programming/debugging that hinders these solvers' broad impact. In the paper, the library skscope is introduced to overcome such an obstacle. With skscope, users can solve the SCO by just programming the objective function. The convenience of skscope is demonstrated through two examples in the paper, where sparse linear regression and trend filtering are addressed with just …

This https://arxiv.org/abs/2206.03975 has been replaced.
link: https://scholar.google.com/scholar?q=a

Functional linear and single-index models: A unified approach via Gaussian Stein identity
Functional linear and single-index models are core regression methods in functional data analysis and are widely used for performing regression in a wide range of applications when the covariates are random functions coupled with scalar responses. In the existing literature, however, the construction of associated estimators and the study of their theoretical properties is invariably carried out on a case-by-case basis for specific models under consideration. In this work, assuming the predicto…

Exploring the Links between the Fundamental Lemma and Kernel Regression
Oleksii Molodchyk, Timm Faulwasser
https://arxiv.org/abs/2403.05368 https://…

Exploring the Links between the Fundamental Lemma and Kernel Regression
Generalizations and variations of the fundamental lemma by Willems et al. are an active topic of recent research. In this note, we explore and formalize the links between kernel regression and known nonlinear extensions of the fundamental lemma. Applying a transformation to the usual linear equation in Hankel matrices, we arrive at an alternative implicit kernel representation of the system trajectories while keeping the requirements on persistency of excitation. We show that this representatio…

A Type of Nonlinear Fr\'echet Regressions
Lu Lin, Ze Chen
https://arxiv.org/abs/2403.17481 https://arxiv.org/pdf/2403.17481

A Type of Nonlinear Fréchet Regressions
The existing Fréchet regression is actually defined within a linear framework, since the weight function in the Fréchet objective function is linearly defined, and the resulting Fréchet regression function is identified to be a linear model when the random object belongs to a Hilbert space. Even for nonparametric and semiparametric Fréchet regressions, which are usually nonlinear, the existing methods handle them by local linear (or local polynomial) technique, and the resulting Fréchet re…

Overparameterized Multiple Linear Regression as Hyper-Curve Fitting
E. Atza, N. Budko
https://arxiv.org/abs/2404.07849 https://arxiv.…

Overparameterized Multiple Linear Regression as Hyper-Curve Fitting
The paper shows that the application of the fixed-effect multiple linear regression model to an overparameterized dataset is equivalent to fitting the data with a hyper-curve parameterized by a single scalar parameter. This equivalence allows for a predictor-focused approach, where each predictor is described by a function of the chosen parameter. It is proven that a linear model will produce exact predictions even in the presence of nonlinear dependencies that violate the model assumptions. Pa…

Sample-Efficient Linear Regression with Self-Selection Bias
Jason Gaitonde, Elchanan Mossel
https://arxiv.org/abs/2402.14229 https://…

Sample-Efficient Linear Regression with Self-Selection Bias
We consider the problem of linear regression with self-selection bias in the unknown-index setting, as introduced in recent work by Cherapanamjeri, Daskalakis, Ilyas, and Zampetakis [STOC 2023]. In this model, one observes $m$ i.i.d. samples $(\mathbf{x}_{\ell},z_{\ell})_{\ell=1}^m$ where $z_{\ell}=\max_{i\in [k]}\{\mathbf{x}_{\ell}^T\mathbf{w}_i+η_{i,\ell}\}$, but the maximizing index $i_{\ell}$ is unobserved. Here, the $\mathbf{x}_{\ell}$ are assumed to be $\mathcal{N}(0,I_n)$ and the noise …

How Well Can Transformers Emulate In-context Newton's Method?
Angeliki Giannou, Liu Yang, Tianhao Wang, Dimitris Papailiopoulos, Jason D. Lee
https://arxiv.org/abs/2403.03183 …

How Well Can Transformers Emulate In-context Newton's Method?
Transformer-based models have demonstrated remarkable in-context learning capabilities, prompting extensive research into its underlying mechanisms. Recent studies have suggested that Transformers can implement first-order optimization algorithms for in-context learning and even second order ones for the case of linear regression. In this work, we study whether Transformers can perform higher order optimization methods, beyond the case of linear regression. We establish that linear attention Tr…

Semiparametric Inference for Regression-Discontinuity Designs
Rong J. B. Zhu, Weiwei Jiang
https://arxiv.org/abs/2403.05803 https://a…

Semiparametric Inference for Regression-Discontinuity Designs
Treatment effects in regression discontinuity designs (RDDs) are often estimated using local regression methods. However, global approximation methods are generally deemed inefficient. In this paper, we propose a semiparametric framework tailored for estimating treatment effects in RDDs. Our global approach conceptualizes the identification of treatment effects within RDDs as a partially linear modeling problem, with the linear component capturing the treatment effect. Furthermore, we utilize t…

This https://arxiv.org/abs/2403.20200 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

High-dimensional analysis of ridge regression for non-identically distributed data with a variance profile
High-dimensional linear regression has been thoroughly studied in the context of independent and identically distributed data. We propose to investigate high-dimensional regression models for independent but non-identically distributed data. To this end, we suppose that the set of observed predictors (or features) is a random matrix with a variance profile and with dimensions growing at a proportional rate. Assuming a random effect model, we study the predictive risk of the ridge estimator for …

Inferring Change Points in High-Dimensional Linear Regression via Approximate Message Passing
Gabriel Arpino, Xiaoqi Liu, Ramji Venkataramanan
https://arxiv.org/abs/2404.07864

Inferring Change Points in High-Dimensional Linear Regression via Approximate Message Passing
We consider the problem of localizing change points in high-dimensional linear regression. We propose an Approximate Message Passing (AMP) algorithm for estimating both the signals and the change point locations. Assuming Gaussian covariates, we give an exact asymptotic characterization of its estimation performance in the limit where the number of samples grows proportionally to the signal dimension. Our algorithm can be tailored to exploit any prior information on the signal, noise, and chang…

This https://arxiv.org/abs/2403.20200 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

High-dimensional analysis of ridge regression for non-identically distributed data with a variance profile
High-dimensional linear regression has been thoroughly studied in the context of independent and identically distributed data. We propose to investigate high-dimensional regression models for independent but non-identically distributed data. To this end, we suppose that the set of observed predictors (or features) is a random matrix with a variance profile and with dimensions growing at a proportional rate. Assuming a random effect model, we study the predictive risk of the ridge estimator for …

This https://arxiv.org/abs/2302.03157 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

A distribution-free mixed-integer optimization approach to hierarchical modelling of clustered and longitudinal data
Recent advancements in Mixed Integer Optimization (MIO) algorithms, paired with hardware enhancements, have led to significant speedups in resolving MIO problems. These strategies have been utilized for optimal subset selection, specifically for choosing $k$ features out of $p$ in linear regression given $n$ observations. In this paper, we broaden this method to facilitate cluster-aware regression, where selection aims to choose $λ$ out of $K$ clusters in a linear mixed effects (LMM) model wit…

This https://arxiv.org/abs/2306.10529 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Dropout Regularization Versus $\ell_2$-Penalization in the Linear Model
We investigate the statistical behavior of gradient descent iterates with dropout in the linear regression model. In particular, non-asymptotic bounds for the convergence of expectations and covariance matrices of the iterates are derived. The results shed more light on the widely cited connection between dropout and l2-regularization in the linear model. We indicate a more subtle relationship, owing to interactions between the gradient descent dynamics and the additional randomness induced by …

Multi-fidelity Gaussian process surrogate modeling for regression problems in physics
Kislaya Ravi, Vladyslav Fediukov, Felix Dietrich, Tobias Neckel, Fabian Buse, Michael Bergmann, Hans-Joachim Bungartz
https://arxiv.org/abs/2404.11965

Multi-fidelity Gaussian process surrogate modeling for regression problems in physics
One of the main challenges in surrogate modeling is the limited availability of data due to resource constraints associated with computationally expensive simulations. Multi-fidelity methods provide a solution by chaining models in a hierarchy with increasing fidelity, associated with lower error, but increasing cost. In this paper, we compare different multi-fidelity methods employed in constructing Gaussian process surrogates for regression. Non-linear autoregressive methods in the existing l…

This https://arxiv.org/abs/2306.10529 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Dropout Regularization Versus $\ell_2$-Penalization in the Linear Model
We investigate the statistical behavior of gradient descent iterates with dropout in the linear regression model. In particular, non-asymptotic bounds for the convergence of expectations and covariance matrices of the iterates are derived. The results shed more light on the widely cited connection between dropout and l2-regularization in the linear model. We indicate a more subtle relationship, owing to interactions between the gradient descent dynamics and the additional randomness induced by …

This https://arxiv.org/abs/2303.08790 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

lmw: Linear Model Weights for Causal Inference
The linear regression model is widely used in the biomedical and social sciences as well as in policy and business research to adjust for covariates and estimate the average effects of treatments. Behind every causal inference endeavor there is a hypothetical randomized experiment. However, in routine regression analyses in observational studies, it is unclear how well the adjustments made by regression approximate key features of randomized experiments, such as covariate balance, study represe…

Local-Polynomial Estimation for Multivariate Regression Discontinuity Designs
Masayuki Sawada, Takuya Ishihara, Daisuke Kurisu, Yasumasa Matsuda
https://arxiv.org/abs/2402.08941 <…

Local-Polynomial Estimation for Multivariate Regression Discontinuity Designs
We introduce a multivariate local-linear estimator for multivariate regression discontinuity designs in which treatment is assigned by crossing a boundary in the space of running variables. The dominant approach uses the Euclidean distance from a boundary point as the scalar running variable; hence, multivariate designs are handled as uni-variate designs. However, the distance running variable is incompatible with the assumption for asymptotic validity. We handle multivariate designs as multiva…

This https://arxiv.org/abs/2310.08391 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?
Transformers pretrained on diverse tasks exhibit remarkable in-context learning (ICL) capabilities, enabling them to solve unseen tasks solely based on input contexts without adjusting model parameters. In this paper, we study ICL in one of its simplest setups: pretraining a linearly parameterized single-layer linear attention model for linear regression with a Gaussian prior. We establish a statistical task complexity bound for the attention model pretraining, showing that effective pretrainin…

Symmetry: a General Structure in Nonparametric Regression
Louis G. Christie, John A. D. Aston
https://arxiv.org/abs/2404.12943 https://

Symmetry: a General Structure in Nonparametric Regression
In this paper we present the framework of symmetry in nonparametric regression. This generalises the framework of covariate sparsity, where the regression function depends only on at most $s < d$ of the covariates, which is a special case of translation symmetry with linear orbits. In general this extends to other types of functions that capture lower dimensional behavior even when these structures are non-linear. We show both that known symmetries of regression functions can be exploited to gi…

Interval-censored linear quantile regression
Taehwa Choi, Seohyeon Park, Hunyong Cho, Sangbum Choi
https://arxiv.org/abs/2404.11125 https://

Interval-censored linear quantile regression
Censored quantile regression has emerged as a prominent alternative to classical Cox's proportional hazards model or accelerated failure time model in both theoretical and applied statistics. While quantile regression has been extensively studied for right-censored survival data, methodologies for analyzing interval-censored data remain limited in the survival analysis literature. This paper introduces a novel local weighting approach for estimating linear censored quantile regression, specific…

This https://arxiv.org/abs/2309.06429 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Efficient Inference on High-Dimensional Linear Models with Missing Outcomes
This paper is concerned with inference on the regression function of a high-dimensional linear model when outcomes are missing at random. We propose an estimator which combines a Lasso pilot estimate of the regression function with a bias correction term based on the weighted residuals of the Lasso regression. The weights depend on estimates of the missingness probabilities (propensity scores) and solve a convex optimization program that trades off bias and variance optimally. Provided that the…

Debiased LASSO under Poisson-Gauss Model
Pedro Abdalla, Gil Kur
https://arxiv.org/abs/2402.16764 https://arxiv.org/pdf/2402.16764

Debiased LASSO under Poisson-Gauss Model
Quantifying uncertainty in high-dimensional sparse linear regression is a fundamental task in statistics that arises in various applications. One of the most successful methods for quantifying uncertainty is the debiased LASSO, which has a solid theoretical foundation but is restricted to settings where the noise is purely additive. Motivated by real-world applications, we study the so-called Poisson inverse problem with additive Gaussian noise and propose a debiased LASSO algorithm that only r…

This https://arxiv.org/abs/2211.04034 has been replaced.
link: https://scholar.google.com/scholar?q=a

Structured Mixture of Continuation-ratio Logits Models for Ordinal Regression
We develop a nonparametric Bayesian modeling approach to ordinal regression based on priors placed directly on the discrete distribution of the ordinal responses. The prior probability models are built from a structured mixture of multinomial distributions. We leverage a continuation-ratio logits representation to formulate the mixture kernel, with mixture weights defined through the logit stick-breaking process that incorporates the covariates through a linear function. The implied regression …

Nonparametric Regression under Cluster Sampling
Yuya Shimizu
https://arxiv.org/abs/2403.04766 https://arxiv.org/pdf/2403.04766…

Nonparametric Regression under Cluster Sampling
This paper develops a general asymptotic theory for nonparametric kernel regression in the presence of cluster dependence. We examine nonparametric density estimation, Nadaraya-Watson kernel regression, and local linear estimation. Our theory accommodates growing and heterogeneous cluster sizes. We derive asymptotic conditional bias and variance, establish uniform consistency, and prove asymptotic normality. Our findings reveal that under heterogeneous cluster sizes, the asymptotic variance inc…

Failures and Successes of Cross-Validation for Early-Stopped Gradient Descent
Pratik Patil, Yuchen Wu, Ryan J. Tibshirani
https://arxiv.org/abs/2402.16793 …

Failures and Successes of Cross-Validation for Early-Stopped Gradient Descent
We analyze the statistical properties of generalized cross-validation (GCV) and leave-one-out cross-validation (LOOCV) applied to early-stopped gradient descent (GD) in high-dimensional least squares regression. We prove that GCV is generically inconsistent as an estimator of the prediction risk of early-stopped GD, even for a well-specified linear model with isotropic features. In contrast, we show that LOOCV converges uniformly along the GD trajectory to the prediction risk. Our theory requir…

Robust Numerical Methods for Nonlinear Regression
Peng Liu, William Q. Meeker
https://arxiv.org/abs/2403.12759 https://arxiv.org/pdf/…

Robust Numerical Methods for Nonlinear Regression
Many scientific and engineering applications require fitting regression models that are nonlinear in the parameters. Advances in computer hardware and software in recent decades have made it easier to fit such models. Relative to fitting regression models that are linear in the parameters, however, fitting nonlinear regression models is more complicated. In particular, software like the $\texttt{nls}$ R function requires care in how the model is parameterized and how initial values are chosen f…

On estimation of heavy-tailed stable linear regression
Eitaro Kawamo, Hiroki Masuda
https://arxiv.org/abs/2404.10448 https://arxiv.or…

On estimation of heavy-tailed stable linear regression
We study the parameter estimation method for linear regression models with possibly skewed stable distributed errors. Our estimation procedure consists of two stages: first, for the regression coefficients, the Cauchy quasi-maximum likelihood estimator (CQMLE) is considered after taking the differences to remove the skewness of noise, and we prove its asymptotic normality and tail-probability estimate; second, as for stable-distribution parameters, we consider the moment estimators based on the…

This https://arxiv.org/abs/2010.02848 has been replaced.
link: https://scholar.google.com/scholar?q=a

Unified Robust Estimation
Robust estimation is primarily concerned with providing reliable parameter estimates in the presence of outliers. Numerous robust loss functions have been proposed in regression and classification, along with various computing algorithms. In modern penalised generalised linear models (GLM), however, there is limited research on robust estimation that can provide weights to determine the outlier status of the observations. This article proposes a unified framework based on a large family of loss…

Detection and inference of changes in high-dimensional linear regression with non-sparse structures
Haeran Cho, Tobias Kley, Hounsen Li
https://arxiv.org/abs/2402.06915

Detection and inference of changes in high-dimensional linear regression with non-sparse structures
For the data segmentation problem in high-dimensional linear regression settings, a commonly made assumption is that the regression parameters are segment-wise sparse, which enables many existing methods to estimate the parameters locally via $\ell_1$-regularised maximum likelihood-type estimation and contrast them for change point detection. Contrary to the common belief, we show that the sparsity of neither regression parameters nor their differences, a.k.a.\ differential parameters, is neces…

Optimal convex $M$-estimation via score matching
Oliver Y. Feng, Yu-Chun Kao, Min Xu, Richard J. Samworth
https://arxiv.org/abs/2403.16688 https://<…

Optimal convex $M$-estimation via score matching
In the context of linear regression, we construct a data-driven convex loss function with respect to which empirical risk minimisation yields optimal asymptotic variance in the downstream estimation of the regression coefficients. Our semiparametric approach targets the best decreasing approximation of the derivative of the log-density of the noise distribution. At the population level, this fitting process is a nonparametric extension of score matching, corresponding to a log-concave projectio…

Analysing heavy-tail properties of Stochastic Gradient Descent by means of Stochastic Recurrence Equations
Ewa Damek, Sebastian Mentemeier
https://arxiv.org/abs/2403.13868

Analysing heavy-tail properties of Stochastic Gradient Descent by means of Stochastic Recurrence Equations
In recent works on the theory of machine learning, it has been observed that heavy tail properties of Stochastic Gradient Descent (SGD) can be studied in the probabilistic framework of stochastic recursions. In particular, Gürbüzbalaban et al. (arXiv:2006.04740) considered a setup corresponding to linear regression for which iterations of SGD can be modelled by a multivariate affine stochastic recursion $X_k=A_k X_{k-1}+B_k$, for independent and identically distributed pairs $(A_k, B_k)$, whe…

Quadratic inference with dense functional responses
Pratim Guha Niyogi, Ping-Shou Zhong
https://arxiv.org/abs/2402.13907 https://arxi…

Quadratic inference with dense functional responses
We address the challenge of estimation in the context of constant linear effect models with dense functional responses. In this framework, the conditional expectation of the response curve is represented by a linear combination of functional covariates with constant regression parameters. In this paper, we present an alternative solution by employing the quadratic inference approach, a well-established method for analyzing correlated data, to estimate the regression coefficients. Our approach l…

Adaptive Ridge Approach to Heteroscedastic Regression
Ka Long Keith Ho, Hiroki Masuda
https://arxiv.org/abs/2402.13642 https://arxiv.…

Adaptive Ridge Approach to Heteroscedastic Regression
We propose an adaptive ridge (AR) based estimation scheme for a heteroscedastic linear model equipped with log-linear errors. We simultaneously estimate the mean and variance parameters and show new asymptotic distributional and tightness properties in a sparse setting. We also show that estimates for zero parameters shrink with more iterations under suitable assumptions for tuning parameters. We observe possible generalizations of this paper's results through simulations and will apply the est…

This https://arxiv.org/abs/2305.12624 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Scalable regression calibration approaches to correcting measurement error in multi-level generalized functional linear regression models with heteroscedastic measurement errors
Wearable devices permit the continuous monitoring of biological processes, such as blood glucose metabolism, and behavior, such as sleep quality and physical activity. The continuous monitoring often occurs in epochs of 60 seconds over multiple days, resulting in high dimensional longitudinal curves that are best described and analyzed as functional data. From this perspective, the functional data are smooth, latent functions obtained at discrete time intervals and prone to homoscedastic white …

On the probability of linear separability through intrinsic volumes
Felix Kuchelmeister
https://arxiv.org/abs/2404.12889 https://arxi…

On the probability of linear separability through intrinsic volumes
A dataset with two labels is linearly separable if it can be split into its two classes with a hyperplane. This inflicts a curse on some statistical tools (such as logistic regression) but forms a blessing for others (e.g. support vector machines). Recently, the following question has regained interest: What is the probability that the data are linearly separable? We provide a formula for the probability of linear separability for Gaussian features and labels depending only on one marginal of…

This https://arxiv.org/abs/2305.00218 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Subdata selection for big data regression: an improved approach
In the big data era researchers face a series of problems. Even standard approaches/methodologies, like linear regression, can be difficult or problematic with huge volumes of data. Traditional approaches for regression in big datasets may suffer due to the large sample size, since they involve inverting huge data matrices or even because the data cannot fit to the memory. Proposed approaches are based on selecting representative subdata to run the regression. Existing approaches select the sub…

This https://arxiv.org/abs/2310.07852 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

On the Computational Complexity of Private High-dimensional Model Selection
We consider the problem of model selection in a high-dimensional sparse linear regression model under privacy constraints. We propose a differentially private best subset selection method with strong utility properties by adopting the well-known exponential mechanism for selecting the best model. We propose an efficient Metropolis-Hastings algorithm and establish that it enjoys polynomial mixing time to its stationary distribution. Furthermore, we also establish approximate differential privacy…

This https://arxiv.org/abs/2310.07852 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

On the Computational Complexity of Private High-dimensional Model Selection
We consider the problem of model selection in a high-dimensional sparse linear regression model under privacy constraints. We propose a differentially private best subset selection method with strong utility properties by adopting the well-known exponential mechanism for selecting the best model. We propose an efficient Metropolis-Hastings algorithm and establish that it enjoys polynomial mixing time to its stationary distribution. Furthermore, we also establish approximate differential privacy…

A Bayesian shrinkage estimator for transfer learning
Mohamed A. Abba, Jonathan P. Williams, Brian J. Reich
https://arxiv.org/abs/2403.17321 https://…

A Bayesian shrinkage estimator for transfer learning
Transfer learning (TL) has emerged as a powerful tool to supplement data collected for a target task with data collected for a related source task. The Bayesian framework is natural for TL because information from the source data can be incorporated in the prior distribution for the target data analysis. In this paper, we propose and study Bayesian TL methods for the normal-means problem and multiple linear regression. We propose two classes of prior distributions. The first class assumes the d…

Multiple-output composite quantile regression through an optimal transport lens
Xuzhi Yang, Tengyao Wang
https://arxiv.org/abs/2402.09098 https://

Multiple-output composite quantile regression through an optimal transport lens
Composite quantile regression has been used to obtain robust estimators of regression coefficients in linear models with good statistical efficiency. By revealing an intrinsic link between the composite quantile regression loss function and the Wasserstein distance from the residuals to the set of quantiles, we establish a generalization of the composite quantile regression to the multiple-output settings. Theoretical convergence rates of the proposed estimator are derived both under the settin…

This https://arxiv.org/abs/2304.04712 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Testing for linearity in scalar-on-function regression with responses missing at random
A goodness-of-fit test for the Functional Linear Model with Scalar Response (FLMSR) with responses Missing at Random (MAR) is proposed in this paper. The test statistic relies on a marked empirical process indexed by the projected functional covariate and its distribution under the null hypothesis is calibrated using a wild bootstrap procedure. The computation and performance of the test rely on having an accurate estimator of the functional slope of the FLMSR when the sample has MAR responses.…

Asymptotic Theory for Linear Functionals of Kernel Ridge Regression
Rui Tuo, Lu Zou
https://arxiv.org/abs/2403.04248 https://arxiv.or…

Asymptotic Theory for Linear Functionals of Kernel Ridge Regression
An asymptotic theory is established for linear functionals of the predictive function given by kernel ridge regression, when the reproducing kernel Hilbert space is equivalent to a Sobolev space. The theory covers a wide variety of linear functionals, including point evaluations, evaluation of derivatives, $L_2$ inner products, etc. We establish the upper and lower bounds of the estimates and their asymptotic normality. It is shown that $λ\sim n^{-1}$ is the universal optimal order of magnitud…

Group COMBSS: Group Selection via Continuous Optimization
Anant Mathur, Sarat Moka, Benoit Liquet, Zdravko Botev
https://arxiv.org/abs/2404.13339 https://<…

Group COMBSS: Group Selection via Continuous Optimization
We present a new optimization method for the group selection problem in linear regression. In this problem, predictors are assumed to have a natural group structure and the goal is to select a small set of groups that best fits the response. The incorporation of group structure in a predictor matrix is a key factor in obtaining better estimators and identifying associations between response and predictors. Such a discrete constrained problem is well-known to be hard, particularly in high-dimens…

Group COMBSS: Group Selection via Continuous Optimization
Anant Mathur, Sarat Moka, Benoit Liquet, Zdravko Botev
https://arxiv.org/abs/2404.13339 https://<…

Group COMBSS: Group Selection via Continuous Optimization
We present a new optimization method for the group selection problem in linear regression. In this problem, predictors are assumed to have a natural group structure and the goal is to select a small set of groups that best fits the response. The incorporation of group structure in a predictor matrix is a key factor in obtaining better estimators and identifying associations between response and predictors. Such a discrete constrained problem is well-known to be hard, particularly in high-dimens…

This https://arxiv.org/abs/2111.11694 has been replaced.
link: https://scholar.google.com/scholar?q=a

MARS via LASSO
Multivariate adaptive regression splines (MARS) is a popular method for nonparametric regression introduced by Friedman in 1991. MARS fits simple nonlinear and non-additive functions to regression data. We propose and study a natural lasso variant of the MARS method. Our method is based on least squares estimation over a convex class of functions obtained by considering infinite-dimensional linear combinations of functions in the MARS basis and imposing a variation based complexity constraint. …

Robust inference for linear regression models with possibly skewed error distribution
Amarnath Nandy, Ayanendranath Basu, Abhik Ghosh
https://arxiv.org/abs/2404.03404

Robust inference for linear regression models with possibly skewed error distribution
Traditional methods for linear regression generally assume that the underlying error distribution, equivalently the distribution of the responses, is normal. Yet, sometimes real life response data may exhibit a skewed pattern, and assuming normality would not give reliable results in such cases. This is often observed in cases of some biomedical, behavioral, socio-economic and other variables. In this paper, we propose to use the class of skew normal (SN) distributions, which also includes the …

Estimating the linear relation between variables that are never jointly observed: an application in in vivo experiments
Polina Arsenteva, Mohamed Amine Benadjaoud, Herv\'e Cardot
https://arxiv.org/abs/2403.00140

Estimating the linear relation between variables that are never jointly observed: an application in in vivo experiments
This work is motivated by in vivo experiments in which measurement are destructive so that the variables of interest can never be observed simultaneously when the aim is to estimate the regression coefficients of a linear regression. Assuming that the global experiment can be decomposed into sub experiments (corresponding for example to different doses) with distinct first moments, we propose different estimators of the linear regression which take account of that additional information. We con…

This https://arxiv.org/abs/2402.06915 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Detection and inference of changes in high-dimensional linear regression with non-sparse structures
For data segmentation in high-dimensional linear regression settings, the regression parameters are often assumed to be sparse segment-wise, which enables many existing methods to estimate the parameters locally via $\ell_1$-regularised maximum likelihood-type estimation and then contrast them for change point detection. Contrary to this common practice, we show that the sparsity of neither regression parameters nor their differences, a.k.a. differential parameters, is necessary for consistency…

Inference for non-stationary time series quantile regression with inequality constraints
Yuan Sun, Zhou Zhou
https://arxiv.org/abs/2404.03837 https://

Inference for non-stationary time series quantile regression with inequality constraints
We consider parameter inference for linear quantile regression with non-stationary predictors and errors, where the regression parameters are subject to inequality constraints. We show that the constrained quantile coefficient estimators are asymptotically equivalent to the metric projections of the unconstrained estimator onto the constrained parameter space. Utilizing a geometry-invariant property of this projection operation, we propose inference procedures - the Wald, likelihood ratio, and …

This https://arxiv.org/abs/2207.04773 has been replaced.
link: https://scholar.google.com/scholar?q=a

Functional Regression Models with Functional Response: New Approaches and a Comparative Study
This paper proposes a new nonlinear approach for additive functional regression with functional response based on kernel methods along with some slight reformulation and implementation of the linear regression and the spectral additive model. The latter methods have in common that the covariates and the response are represented in a basis and so, can only be applied when the response and the covariates belong to a Hilbert space, while the proposed method only uses the distances among data and t…

Functional Partial Least-Squares: Optimal Rates and Adaptation
Andrii Babii, Marine Carrasco, Idriss Tsafack
https://arxiv.org/abs/2402.11134 https://

Functional Partial Least-Squares: Optimal Rates and Adaptation
We consider the functional linear regression model with a scalar response and a Hilbert space-valued predictor, a well-known ill-posed inverse problem. We propose a new formulation of the functional partial least-squares (PLS) estimator related to the conjugate gradient method. We shall show that the estimator achieves the (nearly) optimal convergence rate on a class of ellipsoids and we introduce an early stopping rule which adapts to the unknown degree of ill-posedness. Some theoretical and s…

This https://arxiv.org/abs/2207.04773 has been replaced.
link: https://scholar.google.com/scholar?q=a

Functional Regression Models with Functional Response: New Approaches and a Comparative Study
This paper proposes a new nonlinear approach for additive functional regression with functional response based on kernel methods along with some slight reformulation and implementation of the linear regression and the spectral additive model. The latter methods have in common that the covariates and the response are represented in a basis and so, can only be applied when the response and the covariates belong to a Hilbert space, while the proposed method only uses the distances among data and t…

This https://arxiv.org/abs/2303.17823 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

An interpretable neural network-based non-proportional odds model for ordinal regression
This study proposes an interpretable neural network-based non-proportional odds model (N$^3$POM) for ordinal regression. N$^3$POM is different from conventional approaches to ordinal regression with non-proportional models in several ways: (1) N$^3$POM is defined for both continuous and discrete responses, whereas standard methods typically treat the ordered continuous variables as if they are discrete, (2) instead of estimating response-dependent finite-dimensional coefficients of linear model…

Statistical tests for comparing the associations of multiple exposures with a common outcome in Cox proportional hazard models
Rikuta Hamaya, Peilu Wang, Lin Ge, Edward L. Giovannucci, Molin Wang
https://arxiv.org/abs/2403.14044

Statistical tests for comparing the associations of multiple exposures with a common outcome in Cox proportional hazard models
With advancement of medicine, alternative exposures or interventions are emerging with respect to a common outcome, and there are needs to formally test the difference in the associations of multiple exposures. We propose a duplication method-based multivariate Wald test in the Cox proportional hazard regression analyses to test the difference in the associations of multiple exposures with a same outcome. The proposed method applies to linear or categorical exposures. To illustrate our method, …

Causal Change Point Detection and Localization
Shimeng Huang, Jonas Peters, Niklas Pfister
https://arxiv.org/abs/2403.12677 https://a…

Causal Change Point Detection and Localization
Detecting and localizing change points in sequential data is of interest in many areas of application. Various notions of change points have been proposed, such as changes in mean, variance, or the linear regression coefficient. In this work, we consider settings in which a response variable $Y$ and a set of covariates $X=(X^1,\ldots,X^{d+1})$ are observed over time and aim to find changes in the causal mechanism generating $Y$ from $X$. More specifically, we assume $Y$ depends linearly on a su…

Effects of model misspecification on small area estimators
Yuting Chen, Partha Lahiri, Nicola Salvati
https://arxiv.org/abs/2403.11276 https://

Effects of model misspecification on small area estimators
Nested error regression models are commonly used to incorporate observational unit specific auxiliary variables to improve small area estimates. When the mean structure of this model is misspecified, there is generally an increase in the mean square prediction error (MSPE) of Empirical Best Linear Unbiased Predictors (EBLUP). Observed Best Prediction (OBP) method has been proposed with the intent to improve on the MSPE over EBLUP. We conduct a Monte Carlo simulation experiment to understand the…

This https://arxiv.org/abs/2312.00130 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Sparse Data-Driven Random Projection in Regression for High-Dimensional Data
We examine the linear regression problem in a challenging high-dimensional setting with correlated predictors where the vector of coefficients can vary from sparse to dense. In this setting, we propose a combination of probabilistic variable screening with random projection tools as a viable approach. More specifically, we introduce a new data-driven random projection tailored to the problem at hand and derive a theoretical bound on the gain in expected prediction error over conventional random…

Estimation of Quantile Functionals in Linear Model
Jana Jure\v{c}kov\'a, Jan Picek, Jan Kalina
https://arxiv.org/abs/2404.02764 https://

Estimation of Quantile Functionals in Linear Model
Various indicators and measures of the real life procedures rise up as functionals of the quantile process of a parent random variable Z. However, Z can be observed only through a response in a linear model whose covariates are not under our control and the probability distribution of error terms is generally unknown. The problem is that of nonparametric estimation or other inference for such functionals. We propose an estimation procedure based on the averaged two-step regression quantile, rec…

An Approximation Based Theory of Linear Regression
Laurie Davies
https://arxiv.org/abs/2402.09858 https://arxiv.org/pdf/2402.09858