Tootfinder

A Systematic Literature Survey of Sparse Matrix-Vector Multiplication
Jianhua Gao, Bingjie Liu, Weixing Ji, Hua Huang
https://arxiv.org/abs/2404.06047 http…

A Systematic Literature Survey of Sparse Matrix-Vector Multiplication
Sparse matrix-vector multiplication (SpMV) is a crucial computing kernel with widespread applications in iterative algorithms. Over the past decades, research on SpMV optimization has made remarkable strides, giving rise to various optimization contributions. However, the comprehensive and systematic literature survey that introduces, analyzes, discusses, and summarizes the advancements of SpMV in recent years is currently lacking. Aiming to fill this gap, this paper compares existing technique…

This https://arxiv.org/abs/2402.03216 has been replaced.
link: https://scholar.google.com/scholar?q=a

BGE M3-Embedding: Multi-Lingual, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation
In this paper, we present a new embedding model, called M3-Embedding, which is distinguished for its versatility in Multi-Linguality, Multi-Functionality, and Multi-Granularity. It can support more than 100 working languages, leading to new state-of-the-art performances on multi-lingual and cross-lingual retrieval tasks. It can simultaneously perform the three common retrieval functionalities of embedding model: dense retrieval, multi-vector retrieval, and sparse retrieval, which provides a uni…

Efficient Sparse Processing-in-Memory Architecture (ESPIM) for Machine Learning Inference
Mingxuan He, Mithuna Thottethodi, T. N. Vijaykumar
https://arxiv.org/abs/2404.04708

Efficient Sparse Processing-in-Memory Architecture (ESPIM) for Machine Learning Inference
Emerging machine learning (ML) models (e.g., transformers) involve memory pin bandwidth-bound matrix-vector (MV) computation in inference. By avoiding pin crossings, processing in memory (PIM) can improve performance and energy for pin-bound workloads, as evidenced by recent commercial efforts in (digital) PIM. Sparse models can improve performance and energy of inference without losing much accuracy. However, unstructured sparse inference injects the key challenges of uncertainty, irregularity…

This https://arxiv.org/abs/2312.00130 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Sparse Data-Driven Random Projection in Regression for High-Dimensional Data
We examine the linear regression problem in a challenging high-dimensional setting with correlated predictors where the vector of coefficients can vary from sparse to dense. In this setting, we propose a combination of probabilistic variable screening with random projection tools as a viable approach. More specifically, we introduce a new data-driven random projection tailored to the problem at hand and derive a theoretical bound on the gain in expected prediction error over conventional random…

Certifying Euclidean Sections and Finding Planted Sparse Vectors Beyond the $\sqrt{n}$ Dimension Threshold
Venkatesan Guruswami, Jun-Ting Hsieh, Prasad Raghavendra
https://arxiv.org/abs/2405.05373

Certifying Euclidean Sections and Finding Planted Sparse Vectors Beyond the $\sqrt{n}$ Dimension Threshold
We consider the task of certifying that a random $d$-dimensional subspace $X$ in $\mathbb{R}^n$ is well-spread - every vector $x \in X$ satisfies $c\sqrt{n} \|x\|_2 \leq \|x\|_1 \leq \sqrt{n}\|x\|_2$. In a seminal work, Barak et. al. showed a polynomial-time certification algorithm when $d \leq O(\sqrt{n})$. On the other hand, when $d \gg \sqrt{n}$, the certification task is information-theoretically possible but there is evidence that it is computationally hard [M…

This https://arxiv.org/abs/2403.09081 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Sparse maximum likelihood estimation for regression models
For regression model selection via maximum likelihood estimation, we adopt a vector representation of candidate models and study the likelihood ratio confidence region for the regression parameter vector of a full model. We show that when its confidence level increases with the sample size at a certain speed, with probability tending to one, the confidence region consists of vectors representing models containing all active variables, including the true parameter vector of the full model. Using…

Xabclib:A Fully Auto-tuned Sparse Iterative Solver
Takahiro Katagiri, Takao Sakurai, Mitsuyoshi Igai, Shoji Itoh, Satoshi Ohshima, Hisayasu Kuroda, Ken Naono, Kengo Nakajima
https://arxiv.org/abs/2405.01599

Xabclib:A Fully Auto-tuned Sparse Iterative Solver
In this paper, we propose a general application programming interface named OpenATLib for auto-tuning (AT). OpenATLib is designed to establish the reusability of AT functions. By using OpenATLib, we develop a fully auto-tuned sparse iterative solver named Xabclib. Xabclib has several novel run-time AT functions. First, the following new implementations of sparse matrix-vector multiplication (SpMV) for thread processing are implemented:(1) non-zero elements; (2) omission of zero-elements computa…

This https://arxiv.org/abs/2405.03708 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

Delta Tensor: Efficient Vector and Tensor Storage in Delta Lake
The exponential growth of artificial intelligence (AI) and machine learning (ML) applications has necessitated the development of efficient storage solutions for vector and tensor data. This paper presents a novel approach for tensor storage in a Lakehouse architecture using Delta Lake. By adopting the multidimensional array storage strategy from array databases and sparse encoding methods to Delta Lake tables, experiments show that this approach has demonstrated notable improvements in both sp…

Sparse Linear Regression and Lattice Problems
Aparna Gupte, Neekon Vafa, Vinod Vaikuntanathan
https://arxiv.org/abs/2402.14645 https://

Sparse Linear Regression and Lattice Problems
Sparse linear regression (SLR) is a well-studied problem in statistics where one is given a design matrix $X\in\mathbb{R}^{m\times n}$ and a response vector $y=Xθ^*+w$ for a $k$-sparse vector $θ^*$ (that is, $\|θ^*\|_0\leq k$) and small, arbitrary noise $w$, and the goal is to find a $k$-sparse $\widehatθ \in \mathbb{R}^n$ that minimizes the mean squared prediction error $\frac{1}{m}\|X\widehatθ-Xθ^*\|^2_2$. While $\ell_1$-relaxation methods such as basis pursuit, Lass…

This https://arxiv.org/abs/2405.03708 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

Delta Tensor: Efficient Vector and Tensor Storage in Delta Lake
The exponential growth of artificial intelligence (AI) and machine learning (ML) applications has necessitated the development of efficient storage solutions for vector and tensor data. This paper presents a novel approach for tensor storage in a Lakehouse architecture using Delta Lake. By adopting the multidimensional array storage strategy from array databases and sparse encoding methods to Delta Lake tables, experiments show that this approach has demonstrated notable improvements in both sp…

Vector Signal Reconstruction Sparse and Parametric Approach of direction of arrival Using Single Vector Hydrophone
Jiabin Guo
https://arxiv.org/abs/2404.15160

Vector Signal Reconstruction Sparse and Parametric Approach of direction of arrival Using Single Vector Hydrophone
This article discusses the application of single vector hydrophones in the field of underwater acoustic signal processing for Direction Of Arrival (DOA) estimation. Addressing the limitations of traditional DOA estimation methods in multi-source environments and under noise interference, this study introduces a Vector Signal Reconstruction Sparse and Parametric Approach (VSRSPA). This method involves reconstructing the signal model of a single vector hydrophone, converting its covariance matrix…

Efficient Unbiased Sparsification
Leighton Barnes, Timothy Chow, Emma Cohen, Keith Frankston, Benjamin Howard, Fred Kochman, Daniel Scheinerman, Jeffrey VanderKam
https://arxiv.org/abs/2402.14925

Efficient Unbiased Sparsification
An unbiased $m$-sparsification of a vector $p\in \mathbb{R}^n$ is a random vector $Q\in \mathbb{R}^n$ with mean $p$ that has at most $m

This https://arxiv.org/abs/2311.00783 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Log-Sum Regularized Kaczmarz Algorithms for High-Order Tensor Recovery
Sparse and low rank tensor recovery has emerged as a significant area of research with applications in many fields such as computer vision. However, minimizing the $\ell_0$-norm of a vector or the rank of a matrix is NP-hard. Instead, their convex relaxed versions are typically adopted in practice due to the computational efficiency, e.g., log-sum penalty. In this work, we propose novel log-sum regularized Kaczmarz algorithms for recovering high-order tensors with either sparse or low-rank stru…

An Analysis on Matching Mechanisms and Token Pruning for Late-interaction Models
Qi Liu, Gang Guo, Jiaxin Mao, Zhicheng Dou, Ji-Rong Wen, Hao Jiang, Xinyu Zhang, Zhao Cao
https://arxiv.org/abs/2403.13291

An Analysis on Matching Mechanisms and Token Pruning for Late-interaction Models
With the development of pre-trained language models, the dense retrieval models have become promising alternatives to the traditional retrieval models that rely on exact match and sparse bag-of-words representations. Different from most dense retrieval models using a bi-encoder to encode each query or document into a dense vector, the recently proposed late-interaction multi-vector models (i.e., ColBERT and COIL) achieve state-of-the-art retrieval effectiveness by using all token embeddings to …

This https://arxiv.org/abs/2209.01172 has been replaced.
link: https://scholar.google.com/scholar?q=a

An Interpretable and Efficient Infinite-Order Vector Autoregressive Model for High-Dimensional Time Series
As a special infinite-order vector autoregressive (VAR) model, the vector autoregressive moving average (VARMA) model can capture much richer temporal patterns than the widely used finite-order VAR model. However, its practicality has long been hindered by its non-identifiability, computational intractability, and difficulty of interpretation, especially for high-dimensional time series. This paper proposes a novel sparse infinite-order VAR model for high-dimensional time series, which avoids a…

Reconstructing Curves from Sparse Samples on Riemannian Manifolds
Diana Marin, Filippo Maggioli, Simone Melzi, Stefan Ohrhallinger, Michael Wimmer
https://arxiv.org/abs/2404.09661

Reconstructing Curves from Sparse Samples on Riemannian Manifolds
Reconstructing 2D curves from sample points has long been a critical challenge in computer graphics, finding essential applications in vector graphics. The design and editing of curves on surfaces has only recently begun to receive attention, primarily relying on human assistance, and where not, limited by very strict sampling conditions. In this work, we formally improve on the state-of-the-art requirements and introduce an innovative algorithm capable of reconstructing closed curves directly …

Koopman Regularization
Ido Cohen
https://arxiv.org/abs/2403.11302 https://arxiv.org/pdf/2403.11302

Koopman Regularization
Restoration, generalization, and dimensionality reduction of a vector field from samples are the most common and crucial tasks in dynamical system analysis. An optimization-based algorithm to fulfill these tasks is suggested. Given noisy, sparse, or redundant sampled vector fields, the optimization process encapsulates the inherent geometry of the dynamical system derived from the Koopman eigenfunction space. The dynamic geometry is revealed via the exact penalty method, compromising accuracy a…

This https://arxiv.org/abs/2403.02324 has been replaced.
link: https://scholar.google.com/scholar?q=a

Differentially Private Communication of Measurement Anomalies in the Smart Grid
In this paper, we present a framework based on differential privacy (DP) for querying electric power measurements to detect system anomalies or bad data. Our DP approach conceals consumption and system matrix data, while simultaneously enabling an untrusted third party to test hypotheses of anomalies, such as the presence of bad data, by releasing a randomized sufficient statistic for hypothesis-testing. We consider a measurement model corrupted by Gaussian noise and a sparse noise vector repre…

This https://arxiv.org/abs/2202.10566 has been replaced.
link: https://scholar.google.com/scholar?q=a

Efficient Massive Machine Type Communication (mMTC) via AMP
We propose efficient and low-complexity multiuser detection (MUD) algorithms for Gaussian multiple access channel (G-MAC) for short-packet transmission in massive machine type communications. To do so, we first formulate the G-MAC MUD problem as a sparse signal recovery problem and obtain the exact and approximate joint prior distribution of the sparse vector to be recovered. Then, we employ the Bayesian approximate message passing (AMP) algorithms with the optimal separable and non-separable m…

This https://arxiv.org/abs/2303.07854 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Empirical Bayes inference in sparse high-dimensional generalized linear models
High-dimensional linear models have been widely studied, but the developments in high-dimensional generalized linear models, or GLMs, have been slower. In this paper, we propose an empirical or data-driven prior leading to an empirical Bayes posterior distribution which can be used for estimation of and inference on the coefficient vector in a high-dimensional GLM, as well as for variable selection. We prove that our proposed posterior concentrates around the true/sparse coefficient vector at t…

On extremal points for some vectorial total variation seminorms
Kristian Bredies, Jos\'e A. Iglesias, Daniel Walter
https://arxiv.org/abs/2404.12831 ht…

On extremal points for some vectorial total variation seminorms
We consider the set of extremal points of the generalized unit ball induced by gradient total variation seminorms for vector-valued functions on bounded Euclidean domains. These extremal points are central to the understanding of sparse solutions and sparse optimization algorithms for variational regularization problems posed among such functions. For not fully vectorial cases in which either the domain or the target are one dimensional, or the sum of the total variations of each component is u…

This https://arxiv.org/abs/2404.02246 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Matrix-weighted estimates beyond Calderón-Zygmund theory
We investigate matrix-weighted bounds for the sublinear non-kernel operators considered by F. Bernicot, D. Frey, and S. Petermichl. We extend their result to sublinear operators acting upon vector-valued functions. First, we dominate these operators by bilinear convex body sparse forms, adapting a recent general principle due to T. Hytönen. Then we use this domination to derive matrix-weighted bounds, adapting arguments of F. Nazarov, S. Petermichl, S. Treil, and A. Volberg. Our requirements o…

This https://arxiv.org/abs/2303.07854 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Empirical Bayes inference in sparse high-dimensional generalized linear models
High-dimensional linear models have been widely studied, but the developments in high-dimensional generalized linear models, or GLMs, have been slower. In this paper, we propose an empirical or data-driven prior leading to an empirical Bayes posterior distribution which can be used for estimation of and inference on the coefficient vector in a high-dimensional GLM, as well as for variable selection. We prove that our proposed posterior concentrates around the true/sparse coefficient vector at t…

Seer: Predictive Runtime Kernel Selection for Irregular Problems
Ryan Swann, Muhammad Osama, Karthik Sangaiah, Jalal Mahmud
https://arxiv.org/abs/2403.17017

Seer: Predictive Runtime Kernel Selection for Irregular Problems
Modern GPUs are designed for regular problems and suffer from load imbalance when processing irregular data. Prior to our work, a domain expert selects the best kernel to map fine-grained irregular parallelism to a GPU. We instead propose Seer, an abstraction for producing a simple, reproduceable, and understandable decision tree selector model which performs runtime kernel selection for irregular workloads. To showcase our framework, we conduct a case study in Sparse Matrix Vector Multiplicati…

Sparse maximum likelihood estimation for regression models
Min Tsao
https://arxiv.org/abs/2403.09081 https://arxiv.org/pdf/2403.09081…

Sparse maximum likelihood estimation for regression models
For regression model selection under the maximum likelihood framework, we study the likelihood ratio confidence region for the regression parameter vector of a full regression model. We show that, when the confidence level increases with the sample size at a certain speed, with probability tending to one, the confidence region contains only vectors representing models having all active variables, including the parameter vector of the true model. This result leads to a consistent model selection…