Tootfinder

Vector databases give you the infrastructure to store the embeddings, but how those embeddings are made is the most innovative part of all. Join Sonam Pankaj at this year's Berlin Buzzwords for her short talk on metric learning. #bbuzz
https…

This https://arxiv.org/abs/2402.00974 has been replaced.
link: https://scholar.google.com/scholar?q=a

Coxeter embeddings are injective
We show that certain embeddings of Coxeter groups within other Coxeter groups are injective using the notion of Coxeter partitions. Moreover, we study Lusztig's partitions, which are generalizations of Lusztig's admissible maps and Crisp's foldings. We show that they classify the simplest type of Coxeter partitions, whose embeddings of Coxeter groups send each generator to a product of commuting generators. Consequently, these embeddings are also injective, and we prove that they preserve Coxet…

A characterization of Pfaffian embeddings from (2, 3, 5)- into flat (4, 7)-geometries
Benjamin McMillan
https://arxiv.org/abs/2404.14988 https://

A characterization of Pfaffian embeddings from (2, 3, 5)- into flat (4, 7)-geometries
Given two smooth manifolds with tangent subbundle distributions, an embedding is Pfaffian if its differential sends the distribution on the source into the distribution on the target. In this paper, we consider the question of existence of Pfaffian embeddings in the specific case where the source is a (2,3,5)-manifold, the target is the 7-dimensional space of isotropic 2-planes in a 6-dimensional symplectic vector space, and the Pfaffian condition is that the derived 3-distribution on the sourc…

Face embeddings of Archimedean solids
Tommy Murphy, David Weed
https://arxiv.org/abs/2404.15142 https://arxiv.org/pdf/2404.15142

Face embeddings of Archimedean solids
We characterize the Archimedean solids among the convex uniform polyhedra via face embeddings into a regular Tetrahedron. This result has been listed without proof in the literature. As an application, we obtain a lattice packing of $\mathbb{R}^3$ by the truncated Icosahedron whose packing density is notably close to the optimal value obtained in [1].

A Tight Subexponential-time Algorithm for Two-Page Book Embedding
Robert Ganian, Haiko Mueller, Sebastian Ordyniak, Giacomo Paesani, Mateusz Rychlicki
https://arxiv.org/abs/2404.14087

A Tight Subexponential-time Algorithm for Two-Page Book Embedding
A book embedding of a graph is a drawing that maps vertices onto a line and edges to simple pairwise non-crossing curves drawn into pages, which are half-planes bounded by that line. Two-page book embeddings, i.e., book embeddings into 2 pages, are of special importance as they are both NP-hard to compute and have specific applications. We obtain a 2^(O(\sqrt{n})) algorithm for computing a book embedding of an n-vertex graph on two pages -- a result which is asymptotically tight under the Expon…

This https://arxiv.org/abs/2403.04287 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIR_…

DGR: A General Graph Desmoothing Framework for Recommendation via Global and Local Perspectives
Graph Convolutional Networks (GCNs) have become pivotal in recommendation systems for learning user and item embeddings by leveraging the user-item interaction graph's node information and topology. However, these models often face the famous over-smoothing issue, leading to indistinct user and item embeddings and reduced personalization. Traditional desmoothing methods in GCN-based systems are model-specific, lacking a universal solution. This paper introduces a novel, model-agnostic approach …

A note on the Fourier magnitude data and Sobolev embeddings
Jesse Railo
https://arxiv.org/abs/2404.13329 https://arxiv.org/pdf/2404.1…

A note on the Fourier magnitude data and Sobolev embeddings
We study Sobolev $H^s(\mathbb{R}^n)$, $s \in \mathbb{R}$, stability of the Fourier phase problem to recover $f$ from the knowledge of $|\hat{f}|$ with an additional Bessel potential $H^{t,p}(\mathbb{R}^n)$ a priori estimate when $t \in \mathbb{R}$ and $p \in [1,2]$. These estimates are related to the ones studied recently by Steinerberger in "On the stability of Fourier phase retrieval" J. Fourier Anal. Appl., 28(2):29, 2022. While our estimates in general are different, they share so…

F2FLDM: Latent Diffusion Models with Histopathology Pre-Trained Embeddings for Unpaired Frozen Section to FFPE Translation
Man M. Ho, Shikha Dubey, Yosep Chong, Beatrice Knudsen, Tolga Tasdizen
https://arxiv.org/abs/2404.12650

F2FLDM: Latent Diffusion Models with Histopathology Pre-Trained Embeddings for Unpaired Frozen Section to FFPE Translation
The Frozen Section (FS) technique is a rapid and efficient method, taking only 15-30 minutes to prepare slides for pathologists' evaluation during surgery, enabling immediate decisions on further surgical interventions. However, FS process often introduces artifacts and distortions like folds and ice-crystal effects. In contrast, these artifacts and distortions are absent in the higher-quality formalin-fixed paraffin-embedded (FFPE) slides, which require 2-3 days to prepare. While Generative Ad…

This https://arxiv.org/abs/2308.03104 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_grqc_…

Embedding diagrams in stationary spacetimes
We find the spatial and dynamic embedding diagrams in some stationary spacetimes. The spatial embeddings include the NUT, pure NUT and Kerr spacetimes. In the case of pure NUT spacetime, the spatial embedding equations are solved in terms of the elliptic integrals. In other cases we obtain the spatial embedding diagrams by numerical integration of the corresponding embedding equations. These embedding diagrams are then compared by calculating their Gaussian and mean curvatures. We also find the…

This https://arxiv.org/abs/2403.06849 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

On equivariant embeddings of hyperbolic surfaces into hyperbolic 3-manifolds
We consider the problem of when a closed hyperbolic surface admits a totally geodesic embedding into a closed hyperbolic 3-manifold, and in particular equivariant versions of such embeddings. In a previous paper we considered orientation-preserving actions on orientable surfaces, in the present paper actions of maximal possible order on nonorientable surfaces, and maximal orientation-reversing actions on orientable surfaces.

USE: Dynamic User Modeling with Stateful Sequence Models
Zhihan Zhou, Qixiang Fang, Leonardo Neves, Francesco Barbieri, Yozen Liu, Han Liu, Maarten W. Bos, Ron Dotsch
https://arxiv.org/abs/2403.13344

USE: Dynamic User Modeling with Stateful Sequence Models
User embeddings play a crucial role in user engagement forecasting and personalized services. Recent advances in sequence modeling have sparked interest in learning user embeddings from behavioral data. Yet behavior-based user embedding learning faces the unique challenge of dynamic user modeling. As users continuously interact with the apps, user embeddings should be periodically updated to account for users' recent and long-term behavior patterns. Existing methods highly rely on stateless seq…

This https://arxiv.org/abs/2310.03677 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Embeddings of matrix algebras into uniform Roe algebras and quasi-local algebras
We answer the recent problem posed by Baudier, Braga, Farah, Vignati, and Willett that asks whether the $\ell_\infty$-direct sum of the matrix algebras embeds into the uniform Roe algebra or the quasi-local algebra of a uniformly locally finite metric space. The answers are no and yes, respectively. Hence the inclusion of the uniform Roe algebra into the quasi-local algebra can be proper.

This https://arxiv.org/abs/2404.13569 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSD_…

Musical Word Embedding for Music Tagging and Retrieval
Word embedding has become an essential means for text-based information retrieval. Typically, word embeddings are learned from large quantities of general and unstructured text data. However, in the domain of music, the word embedding may have difficulty understanding musical contexts or recognizing music-related entities like artists and tracks. To address this issue, we propose a new approach called Musical Word Embedding (MWE), which involves learning from various types of texts, including b…

Kernel spectral joint embeddings for high-dimensional noisy datasets using duo-landmark integral operators
Xiucai Ding, Rong Ma
https://arxiv.org/abs/2405.12317

This https://arxiv.org/abs/2212.11165 has been replaced.
link: https://scholar.google.com/scholar?q=a

Distant 2-Colored Components on Embeddings Part III: The General Case
This is the third in a sequence of three papers in which we prove the following generalization of Thomassen's 5-choosability theorem: Let $G$ be a finite graph embedded on a surface of genus $g$. Then $G$ can be $L$-colored, where $L$ is a list-assignment for $G$ in which every vertex has a 5-list except for a collection of pairwise far-apart components, each precolored with an ordinary 2-coloring, as long as the face-width of $G$ is at least $2^{Ω(g)}$ and the precolored components are of dis…

Embeddings transform text into numerical vectors that capture semantic relationships. Join Isabelle Mohr and Bo Wang for their talk at this year's #bbuzz, where they will discuss data preparation, training processes, evaluation and a demo of Jina Embeddings V2 in a hybrid search pipeline, showing its practical applications.

Genetic Programming for Explainable Manifold Learning
Ben Cravens, Andrew Lensen, Paula Maddigan, Bing Xue
https://arxiv.org/abs/2403.14139 https://…

Genetic Programming for Explainable Manifold Learning
Manifold learning techniques play a pivotal role in machine learning by revealing lower-dimensional embeddings within high-dimensional data, thus enhancing both the efficiency and interpretability of data analysis by transforming the data into a lower-dimensional representation. However, a notable challenge with current manifold learning methods is their lack of explicit functional mappings, crucial for explainability in many real-world applications. Genetic programming, known for its interpret…

testing out the @… Custom Javascript this morning and now I'm down a rabbit hole of trying to understand what Embedding numbers are when extracted from page content. You are too? Here is a great piece from @…

This https://arxiv.org/abs/2402.09339 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Robust quasi-isometric embeddings inapproximable by Anosov representations
Let $\mathbb{K}=\mathbb{R}$ or $\mathbb{C}$. For all but finitely many $m\in \mathbb{N}$, we exhibit the first examples of non-locally rigid, Zariski dense, robust quasi-isometric embeddings of hyperbolic groups in $\mathsf{SL}_m(\mathbb{K})$ which are not limits of Anosov representations. As a consequence, we show that an analogue of Sullivan's structural stabilty theorem for Kleinian groups fails for Anosov representations in $\mathsf{SL}_m(\mathbb{C}), m\geq 30$.

CLIP-Embed-KD: Computationally Efficient Knowledge Distillation Using Embeddings as Teachers
Lakshmi Nair
https://arxiv.org/abs/2404.06170 https://<…

CLIP-Embed-KD: Computationally Efficient Knowledge Distillation Using Embeddings as Teachers
Contrastive Language-Image Pre-training (CLIP) has been shown to improve zero-shot generalization capabilities of language and vision models. In this paper, we extend CLIP for efficient knowledge distillation, by utilizing embeddings as teachers. Typical knowledge distillation frameworks require running forward passes through a teacher model, which is often prohibitive in the case of billion or trillion parameter teachers. In these cases, using only the embeddings of the teacher models to guide…

This https://arxiv.org/abs/2312.05049 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

SO$(2,n)$-compatible embeddings of conformally flat $n$-dimensional submanifolds in $\mathbb{R}^{n+2}$
We describe embeddings of $n$-dimensional Lorentzian manifolds, including Friedmann-Lemaître-Robertson-Walker spaces, in $\mathbb{R}^{n+2}$ such that the metrics of the submanifolds are inherited by a restriction from that of $\mathbb{R}^{n+2}$, and the action of the linear group SO$(2, n)$ of the ambient space reduces to conformal transformations on the submanifolds.

This https://arxiv.org/abs/2312.14656 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLO_…

Completions of Kleene's second model
We investigate completions of partial combinatory algebras (pcas), in particular of Kleene's second model $\mathcal{K}_2$ and generalizations thereof. We consider weak and strong notions of embeddability and completion that have been studied before. By a result of Klop it is known that not every pca has a strong completion. The study of completions of $\mathcal{K}_2$ has as corollaries that weak and strong embeddings are different, and that every countable pca has a weak completion. We then con…

Data-Driven Distributionally Robust Safety Verification Using Barrier Certificates and Conditional Mean Embeddings
Oliver Sch\"on, Zhengang Zhong, Sadegh Soudjani
https://arxiv.org/abs/2403.10497

Data-Driven Distributionally Robust Safety Verification Using Barrier Certificates and Conditional Mean Embeddings
Algorithmic verification of realistic systems to satisfy safety and other temporal requirements has suffered from poor scalability of the employed formal approaches. To design systems with rigorous guarantees, many approaches still rely on exact models of the underlying systems. Since this assumption can rarely be met in practice, models have to be inferred from measurement data or are bypassed completely. Whilst former usually requires the model structure to be known a-priori and immense amoun…

This https://arxiv.org/abs/2212.10506 has been replaced.
link: https://scholar.google.com/scholar?q=a

Distant 2-Colored Components on Embeddings Part II: The Short-Inseparable Case
This is the second in a sequence of three papers in which we prove the following generalization of Thomassen's 5-choosability theorem: Let $G$ be a graph embedded on a surface of genus $g$. Then $G$ can be $L$-colored, where $L$ is a list-assignment for $G$ in which every vertex has a 5-list except for a collection of pairwise far-apart components, each precolored with an ordinary 2-coloring, as long as the face-width of $G$ is at least $2^{Ω(g)}$ and the precolored components are of distance …

Stable Coorbit Embeddings of Orbifold Quotients
Yousef Qaddura, Dustin G. Mixon
https://arxiv.org/abs/2403.14042 https://arxiv.org/pd…

Stable Coorbit Embeddings of Orbifold Quotients
Given a real inner product space $V$ and a group $G$ of linear isometries, we construct a family of $G$-invariant real-valued functions on $V$ that we call coorbit filter banks, which unify previous notions of max filter banks and finite coorbit filter banks. When $V=\mathbb R^d$ and $G$ is compact, we establish that a suitable coorbit filter bank is injective and locally lower Lipschitz in the quotient metric at orbits of maximal dimension. Furthermore, when the orbit space $\mathbb S^{d-1}/G$…

This https://arxiv.org/abs/2307.15130 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCG_…

Bounding the Interleaving Distance for Mapper Graphs with a Loss Function
Data consisting of a graph with a function mapping into $\R^d$ arise in many data applications, encompassing structures such as Reeb graphs, geometric graphs, and knot embeddings. As such, the ability to compare and cluster such objects is required in a data analysis pipeline, leading to a need for distances between them. In this work, we study the interleaving distance on discretization of these objects, $\R^d$-mapper graphs, where functor representations of the data can be compared by finding…

Adaptative Bilingual Aligning Using Multilingual Sentence Embedding
Olivier Kraif
https://arxiv.org/abs/2403.11921 https://arxiv.org/…

Adaptative Bilingual Aligning Using Multilingual Sentence Embedding
In this paper, we present an adaptive bitextual alignment system called AIlign. This aligner relies on sentence embeddings to extract reliable anchor points that can guide the alignment path, even for texts whose parallelism is fragmentary and not strictly monotonic. In an experiment on several datasets, we show that AIlign achieves results equivalent to the state of the art, with quasi-linear complexity. In addition, AIlign is able to handle texts whose parallelism and monotonicity properties …

https://www.youtube.com/watch?v=3iZ7jwbAwF8
Embedding Layer of mistral-7b and gemma-7b
Chris Hay plays with the models. Good practical insights with hands on using python.

Graph Vertex Embeddings: Distance, Regularization and Community Detection
Rados{\l}aw Nowak, Adam Ma{\l}kowski, Daniel Cie\'slak, Piotr Sok\'o{\l}, Pawe{\l} Wawrzy\'nski
https://arxiv.org/abs/2404.10784

Graph Vertex Embeddings: Distance, Regularization and Community Detection
Graph embeddings have emerged as a powerful tool for representing complex network structures in a low-dimensional space, enabling the use of efficient methods that employ the metric structure in the embedding space as a proxy for the topological structure of the data. In this paper, we explore several aspects that affect the quality of a vertex embedding of graph-structured data. To this effect, we first present a family of flexible distance functions that faithfully capture the topological dis…

This https://arxiv.org/abs/1204.3191 has been replaced.
link: https://scholar.google.com/scholar?q=a<…

On Isomorphisms of Grassmann Spaces
We exhibit isomorphisms of Grassmann spaces and their relationship with collineations and embeddings of the underlying projective spaces.

Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion
Dongyang Li, Chen Wei, Shiying Li, Jiachen Zou, Quanying Liu
https://arxiv.org/abs/2403.07721

Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion
How to decode human vision through neural signals has attracted a long-standing interest in neuroscience and machine learning. Modern contrastive learning and generative models improved the performance of fMRI-based visual decoding and reconstruction. However, the high cost and low temporal resolution of fMRI limit their applications in brain-computer interfaces (BCIs), prompting a high need for EEG-based visual reconstruction. In this study, we present an EEG-based visual reconstruction framew…

This https://arxiv.org/abs/2403.08206 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIR_…

Discrete Semantic Tokenization for Deep CTR Prediction
Incorporating item content information into click-through rate (CTR) prediction models remains a challenge, especially with the time and space constraints of industrial scenarios. The content-encoding paradigm, which integrates user and item encoders directly into CTR models, prioritizes space over time. In contrast, the embedding-based paradigm transforms item and user semantics into latent embeddings and then caches them, prioritizes space over time. In this paper, we introduce a new semantic…

This https://arxiv.org/abs/2207.12531 has been replaced.
link: https://scholar.google.com/scholar?q=a

Distant 2-Colored Components on Embeddings Part I: Connecting Faces
This is the first in a sequence of three papers in which we prove the following generalization of Thomassen's 5-choosability theorem: Let $G$ be a finite graph embedded on a surface of genus $g$. Then $G$ can be $L$-colored, where $L$ is a list-assignment for $G$ in which every vertex has a 5-list except for a collection of pairwise far-apart components, each precolored with an ordinary 2-coloring, as long as the face-width of $G$ is $2^{Ω(g)}$ and the precolored components are of distance $2^…

Motion Inversion for Video Customization
Luozhou Wang, Guibao Shen, Yixun Liang, Xin Tao, Pengfei Wan, Di Zhang, Yijun Li, Yingcong Chen
https://arxiv.org/abs/2403.20193

Motion Inversion for Video Customization
In this research, we present a novel approach to motion customization in video generation, addressing the widespread gap in the thorough exploration of motion representation within video generative models. Recognizing the unique challenges posed by video's spatiotemporal nature, our method introduces Motion Embeddings, a set of explicit, temporally coherent one-dimensional embeddings derived from a given video. These embeddings are designed to integrate seamlessly with the temporal transformer …

Assessing the Robustness of Spectral Clustering for Deep Speaker Diarization
Nikhil Raghav, Md Sahidullah
https://arxiv.org/abs/2403.14286 https://<…

Assessing the Robustness of Spectral Clustering for Deep Speaker Diarization
Clustering speaker embeddings is crucial in speaker diarization but hasn't received as much focus as other components. Moreover, the robustness of speaker diarization across various datasets hasn't been explored when the development and evaluation data are from different domains. To bridge this gap, this study thoroughly examines spectral clustering for both same-domain and cross-domain speaker diarization. Our extensive experiments on two widely used corpora, AMI and DIHARD, reveal the perform…

Recurrence and Embeddings in Planar Wind-Tree Models
Chen Frenkel
https://arxiv.org/abs/2403.06986 https://arxiv.org/pdf/2403.06986…

Recurrence and Embeddings in Planar Wind-Tree Models
We study periodic infinite billiards in the plane. We show that for rational models, some particular obstacles can be added periodically, so that the billiard flow in the resulting table is recurrent in almost every direction.

Knowledge-Enhanced Recommendation with User-Centric Subgraph Network
Guangyi Liu, Quanming Yao, Yongqi Zhang, Lei Chen
https://arxiv.org/abs/2403.14377 htt…

Knowledge-Enhanced Recommendation with User-Centric Subgraph Network
Recommendation systems, as widely implemented nowadays on various platforms, recommend relevant items to users based on their preferences. The classical methods which rely on user-item interaction matrices has limitations, especially in scenarios where there is a lack of interaction data for new items. Knowledge graph (KG)-based recommendation systems have emerged as a promising solution. However, most KG-based methods adopt node embeddings, which do not provide personalized recommendations for…

Hey everyone!
Due to an uptick in demand we've decided to change the topic next week for ShuttleBytes to "Getting Started with Embeddings & RAG"!
See you there! 🙂
Be subscribed to get notifications when we go live 🔔 http://youtube.com/@shuttle-dev

Process Mining Embeddings: Learning Vector Representations for Petri Nets
Juan G. Colonna, Ahmed A. Fares, M\'arcio Duarte, Ricardo Sousa
https://arxiv.org/abs/2404.17129

Process Mining Embeddings: Learning Vector Representations for Petri Nets
Process mining offers powerful techniques for discovering, analyzing, and enhancing real-world business processes. In this context, Petri nets provide an expressive means of modeling process behavior. However, directly analyzing and comparing intricate Petri net presents challenges. This study introduces PetriNet2Vec, a novel unsupervised methodology based on Natural Language Processing concepts inspired by Doc2Vec and designed to facilitate the effective comparison, clustering, and classificat…

Re-visiting Skip-Gram Negative Sampling: Dimension Regularization for More Efficient Dissimilarity Preservation in Graph Embeddings
David Liu, Arjun Seshadri, Tina Eliassi-Rad, Johan Ugander
https://arxiv.org/abs/2405.00172

Re-visiting Skip-Gram Negative Sampling: Dimension Regularization for More Efficient Dissimilarity Preservation in Graph Embeddings
A wide range of graph embedding objectives decompose into two components: one that attracts the embeddings of nodes that are perceived as similar, and another that repels embeddings of nodes that are perceived as dissimilar. Because real-world graphs are sparse and the number of dissimilar pairs grows quadratically with the number of nodes, Skip-Gram Negative Sampling (SGNS) has emerged as a popular and efficient repulsion approach. SGNS repels each node from a sample of dissimilar nodes, as op…

https://www.youtube.com/watch?v=3iZ7jwbAwF8
Embedding Layer of mistral-7b and gemma-7b
Chris Hay plays with the models. Good practical insights with hands on using python.

Tested the limits of in-browser semantic search with SemanticFinder:
100 books
28.986 pages
158.952 embeddings (Xenova/bge-small-en-v1.5 by @… 1.085.035 lines
54.620.548 characters
Compressed in a ~100Mb .json.gz. 384 dims with 2 decimals. ~3h indexing time on CPU. Test the resulting search performance:

Abelian varieties that split modulo all but finitely many primes
Enric Florit
https://arxiv.org/abs/2404.08496 https://arxiv.org/pdf/…

Abelian varieties that split modulo all but finitely many primes
Let $A$ be a simple abelian variety over a number field $k$ such that $\operatorname{End}(A)$ is noncommutative. We show that $A$ splits modulo all but finitely many primes of $k$. We prove this by considering the subalgebras of $\operatorname{End}(A_{\mathfrak p})\otimes\mathbb{Q}$ which have prime Schur index. Our main tools are Tate's characterization of endomorphism algebras of abelian varieties over finite fields, and a Theorem of Chia-Fu Yu on embeddings of simple algebras.

Stochastic Error Bounds in Nonlinear Model Predictive Control with Gaussian Processes via Parameter-Varying Embeddings
Dimitrios S. Karachalios, Hossam S. Abbas
https://arxiv.org/abs/2405.09209

On equivariant embeddings of hyperbolic surfaces into hyperbolic 3-manifolds
Bruno P. Zimmermann
https://arxiv.org/abs/2403.06849 https://

On equivariant embeddings of hyperbolic surfaces into hyperbolic 3-manifolds
We consider the problem of when a closed hyperbolic surface admits a totally geodesic embedding into a closed hyperbolic 3-manifold, and in particular equivariant versions of such embeddings. In a previous paper we considered orientation-preserving actions on orientable surfaces, in the present paper actions of maximal possible order on nonorientable surfaces, and maximal orientation-reversing actions on orientable surfaces.

This https://arxiv.org/abs/2309.15433 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDB_…

Cardinality Estimation of Subgraph Matching: A Filtering-Sampling Approach
Subgraph counting is a fundamental problem in understanding and analyzing graph structured data, yet computationally challenging. This calls for an accurate and efficient algorithm for Subgraph Cardinality Estimation, which is to estimate the number of all isomorphic embeddings of a query graph in a data graph. We present FaSTest, a novel algorithm that combines (1) a powerful filtering technique to significantly reduce the sample space, (2) an adaptive tree sampling algorithm for accurate and …

This https://arxiv.org/abs/2305.09329 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

CWTM: Leveraging Contextualized Word Embeddings from BERT for Neural Topic Modeling
Most existing topic models rely on bag-of-words (BOW) representation, which limits their ability to capture word order information and leads to challenges with out-of-vocabulary (OOV) words in new documents. Contextualized word embeddings, however, show superiority in word sense disambiguation and effectively address the OOV issue. In this work, we introduce a novel neural topic model called the Contextlized Word Topic Model (CWTM), which integrates contextualized word embeddings from BERT. The…

This https://arxiv.org/abs/2402.17604 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLO_…

Equivariant ideals of polynomials
We study existence and computability of finite bases for ideals of polynomials over infinitely many variables. In our setting, variables come from a countable logical structure A, and embeddings from A to A act on polynomials by renaming variables. First, we give a sufficient and necessary condition for A to guarantee the following generalisation of Hilbert's Basis Theorem: every polynomial ideal which is equivariant, i.e. invariant under renaming of variables, is finitely generated. Second, we…

Compressing and Interpreting Word Embeddings with Latent Space Regularization and Interactive Semantics Probing
Haoyu Li, Junpeng Wang, Yan Zheng, Liang Wang, Wei Zhang, Han-Wei Shen
https://arxiv.org/abs/2403.16815

Compressing and Interpreting Word Embeddings with Latent Space Regularization and Interactive Semantics Probing
Word embedding, a high-dimensional (HD) numerical representation of words generated by machine learning models, has been used for different natural language processing tasks, e.g., translation between two languages. Recently, there has been an increasing trend of transforming the HD embeddings into a latent space (e.g., via autoencoders) for further tasks, exploiting various merits the latent representations could bring. To preserve the embeddings' quality, these works often map the embeddings …

Accelerating Recommender Model Training by Dynamically Skipping Stale Embeddings
Yassaman Ebrahimzadeh Maboud, Muhammad Adnan, Divya Mahajan, Prashant J. Nair
https://arxiv.org/abs/2404.04270

Accelerating Recommender Model Training by Dynamically Skipping Stale Embeddings
Training recommendation models pose significant challenges regarding resource utilization and performance. Prior research has proposed an approach that categorizes embeddings into popular and non-popular classes to reduce the training time for recommendation models. We observe that, even among the popular embeddings, certain embeddings undergo rapid training and exhibit minimal subsequent variation, resulting in saturation. Consequently, updates to these embeddings lack any contribution to mode…

Re-visiting Skip-Gram Negative Sampling: Dimension Regularization for More Efficient Dissimilarity Preservation in Graph Embeddings
David Liu, Arjun Seshadri, Tina Eliassi-Rad, Johan Ugander
https://arxiv.org/abs/2405.00172

Re-visiting Skip-Gram Negative Sampling: Dimension Regularization for More Efficient Dissimilarity Preservation in Graph Embeddings
A wide range of graph embedding objectives decompose into two components: one that attracts the embeddings of nodes that are perceived as similar, and another that repels embeddings of nodes that are perceived as dissimilar. Because real-world graphs are sparse and the number of dissimilar pairs grows quadratically with the number of nodes, Skip-Gram Negative Sampling (SGNS) has emerged as a popular and efficient repulsion approach. SGNS repels each node from a sample of dissimilar nodes, as op…

Birational maps to Grassmannians, representations and poset polytopes
Evgeny Feigin, with an appendix in collaboration with Wojciech Samotij
https://arxiv.org/abs/2403.10074

Birational maps to Grassmannians, representations and poset polytopes
We study the closure of the graph of the birational map from a projective space to a Grassmannian. We provide explicit description of the graph closure and compute the fibers of the natural projection to the Grassmannian. We construct embeddings of the graph closure to the projectivizations of certain cyclic representations of a degenerate special linear Lie algebra and study algebraic and combinatorial properties of these representations. In particular, we describe monomial bases, generalizing…

This https://arxiv.org/abs/2402.12011 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

A Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Change
Contextualized embeddings are the preferred tool for modeling Lexical Semantic Change (LSC). Current evaluations typically focus on a specific task known as Graded Change Detection (GCD). However, performance comparison across work are often misleading due to their reliance on diverse settings. In this paper, we evaluate state-of-the-art models and approaches for GCD under equal conditions. We further break the LSC problem into Word-in-Context (WiC) and Word Sense Induction (WSI) tasks, and com…

This https://arxiv.org/abs/2401.14060 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDS_…

On Sparse Covers of Minor Free Graphs, Low Dimensional Metric Embeddings, and other applications
Given a metric space $(X,d_X)$, a $(β,s,Δ)$-sparse cover is a collection of clusters $\mathcal{C}\subseteq P(X)$ with diameter at most $Δ$, such that for every point $x\in X$, the ball $B_X(x,\fracΔβ)$ is fully contained in some cluster $C\in \mathcal{C}$, and $x$ belongs to at most $s$ clusters in $\mathcal{C}$. Our main contribution is to show that the shortest path metric of every $K_r$-minor free graphs admits $(O(r),O(r^2),Δ)$-sparse cover, and for every $ε>0$, $(4+ε,O(\frac1ε)^r,…

Abelian varieties that split modulo all but finitely many primes
Enric Florit
https://arxiv.org/abs/2404.08496 https://arxiv.org/pdf/…

Abelian varieties that split modulo all but finitely many primes
Let $A$ be a simple abelian variety over a number field $k$ such that $\operatorname{End}(A)$ is noncommutative. We show that $A$ splits modulo all but finitely many primes of $k$. We prove this by considering the subalgebras of $\operatorname{End}(A_{\mathfrak p})\otimes\mathbb{Q}$ which have prime Schur index. Our main tools are Tate's characterization of endomorphism algebras of abelian varieties over finite fields, and a Theorem of Chia-Fu Yu on embeddings of simple algebras.

This https://arxiv.org/abs/2401.14254 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Diagrammatic representations of 3-periodic entanglements
Diagrams enable the use of various algebraic and geometric tools for analysing and classifying knots. In this paper we introduce a new diagrammatic representation of triply-periodic entangled structures (TP tangles), which are embeddings of simple curves in $\mathbb{R}^3$ that are invariant under translations along three non-coplanar axes. As such, these entanglements can be seen as lifts of links in the 3-torus $\mathbb{T}^3 = \mathbb{S}^1 \times \mathbb{S}^1 \times \mathbb{S}^1$, where two no…

Knowledge Graph Completion using Structural and Textual Embeddings
Sakher Khalil Alqaaidi, Krzysztof Kochut
https://arxiv.org/abs/2404.16206 https://

Knowledge Graph Completion using Structural and Textual Embeddings
Knowledge Graphs (KGs) are widely employed in artificial intelligence applications, such as question-answering and recommendation systems. However, KGs are frequently found to be incomplete. While much of the existing literature focuses on predicting missing nodes for given incomplete KG triples, there remains an opportunity to complete KGs by exploring relations between existing nodes, a task known as relation prediction. In this study, we propose a relations prediction model that harnesses bo…

An Analysis on Matching Mechanisms and Token Pruning for Late-interaction Models
Qi Liu, Gang Guo, Jiaxin Mao, Zhicheng Dou, Ji-Rong Wen, Hao Jiang, Xinyu Zhang, Zhao Cao
https://arxiv.org/abs/2403.13291

An Analysis on Matching Mechanisms and Token Pruning for Late-interaction Models
With the development of pre-trained language models, the dense retrieval models have become promising alternatives to the traditional retrieval models that rely on exact match and sparse bag-of-words representations. Different from most dense retrieval models using a bi-encoder to encode each query or document into a dense vector, the recently proposed late-interaction multi-vector models (i.e., ColBERT and COIL) achieve state-of-the-art retrieval effectiveness by using all token embeddings to …

Higher order embeddings via the basepoint-freeness threshold
Federico Caucci
https://arxiv.org/abs/2404.03942 https://arxiv.org/pdf/2…

Higher order embeddings via the basepoint-freeness threshold
In this note, we relate the basepoint-freeness threshold of a polarized abelian variety, introduced by Jiang and Pareschi, with $k$-jet very ampleness. Then, we derive several applications of this fact, including a criterion for the $k$-very ampleness of Kummer varieties.

This https://arxiv.org/abs/2312.07358 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Distributional Bellman Operators over Mean Embeddings
We propose a novel algorithmic framework for distributional reinforcement learning, based on learning finite-dimensional mean embeddings of return distributions. We derive several new algorithms for dynamic programming and temporal-difference learning based on this framework, provide asymptotic convergence theory, and examine the empirical performance of the algorithms on a suite of tabular tasks. Further, we show that this approach can be straightforwardly combined with deep reinforcement lear…

Boone--Higman Embeddings for Contracting Self-Similar Groups
James Belk, Francesco Matucci
https://arxiv.org/abs/2405.10234 https://a…

On Outer Bi-Lipschitz Extensions of Linear Johnson-Lindenstrauss Embeddings of Subsets of $\mathbb{R}^N$
Rafael Chiclana, Mark A. Iwen, Mark Philip Roach
https://arxiv.org/abs/2403.03969

On Outer Bi-Lipschitz Extensions of Linear Johnson-Lindenstrauss Embeddings of Subsets of $\mathbb{R}^N$
The celebrated Johnson-Lindenstrauss lemma states that for all $\varepsilon \in (0,1)$ and finite sets $X \subseteq \mathbb{R}^N$ with $n>1$ elements, there exists a matrix $Φ\in \mathbb{R}^{m \times N}$ with $m=\mathcal{O}(\varepsilon^{-2}\log n)$ such that \[ (1 - \varepsilon) \|x-y\|_2 \leq \|Φx-Φy\|_2 \leq (1+\varepsilon)\| x- y\|_2 \quad \forall\, x, y \in X.\] Herein we consider terminal embedding results which have recently been introduced in the computer…

Cross-lingual Transfer or Machine Translation? On Data Augmentation for Monolingual Semantic Textual Similarity
Sho Hoshino, Akihiko Kato, Soichiro Murakami, Peinan Zhang
https://arxiv.org/abs/2403.05257

Cross-lingual Transfer or Machine Translation? On Data Augmentation for Monolingual Semantic Textual Similarity
Learning better sentence embeddings leads to improved performance for natural language understanding tasks including semantic textual similarity (STS) and natural language inference (NLI). As prior studies leverage large-scale labeled NLI datasets for fine-tuning masked language models to yield sentence embeddings, task performance for languages other than English is often left behind. In this study, we directly compared two data augmentation techniques as potential solutions for monolingual ST…

The total chord length of maximal outerplanar graphs
Haley Broadus, Elena Pavelescu
https://arxiv.org/abs/2404.11028 https://arxiv.or…

The total chord length of maximal outerplanar graphs
We consider embeddings of maximal outerplanar graphs whose vertices all lie on a cycle $\mathcal{C}$ bounding a face. Each edge of the graph that is not in $\mathcal{C}$, a chord, is assigned a length equal to the length of the shortest path in $\mathcal{C}$ between its endpoints. We define the total chord length of a graph as the sum of lengths of all its chords. For each order $n\ge 5$, we find outerplanar graphs whose total chord length is minimal among all graphs of the same order, and grap…

This https://arxiv.org/abs/2403.07721 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csHC_…

Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion
How to decode human vision through neural signals has attracted a long-standing interest in neuroscience and machine learning. Modern contrastive learning and generative models improved the performance of fMRI-based visual decoding and reconstruction. However, the high cost and low temporal resolution of fMRI limit their applications in brain-computer interfaces (BCIs), prompting a high need for EEG-based visual reconstruction. In this study, we present an EEG-based visual reconstruction framew…

Node Similarities under Random Projections: Limits and Pathological Cases
Tvrtko Tadi\'c, Cassiano Becker, Jennifer Neville
https://arxiv.org/abs/2404.10148

Node Similarities under Random Projections: Limits and Pathological Cases
Random Projections have been widely used to generate embeddings for various graph tasks due to their computational efficiency. The majority of applications have been justified through the Johnson-Lindenstrauss Lemma. In this paper, we take a step further and investigate how well dot product and cosine similarity are preserved by Random Projections. Our analysis provides new theoretical results, identifies pathological cases, and tests them with numerical experiments. We find that, for nodes of …

Cross-lingual Transfer or Machine Translation? On Data Augmentation for Monolingual Semantic Textual Similarity
Sho Hoshino, Akihiko Kato, Soichiro Murakami, Peinan Zhang
https://arxiv.org/abs/2403.05257

Cross-lingual Transfer or Machine Translation? On Data Augmentation for Monolingual Semantic Textual Similarity
Learning better sentence embeddings leads to improved performance for natural language understanding tasks including semantic textual similarity (STS) and natural language inference (NLI). As prior studies leverage large-scale labeled NLI datasets for fine-tuning masked language models to yield sentence embeddings, task performance for languages other than English is often left behind. In this study, we directly compared two data augmentation techniques as potential solutions for monolingual ST…

Interpretable Multi-task Learning with Shared Variable Embeddings
Maciej \.Zelaszczyk, Jacek Ma\'ndziuk
https://arxiv.org/abs/2405.06330 https://

Is Cosine-Similarity of Embeddings Really About Similarity?
Harald Steck, Chaitanya Ekanadham, Nathan Kallus
https://arxiv.org/abs/2403.05440 https://

Is Cosine-Similarity of Embeddings Really About Similarity?
Cosine-similarity is the cosine of the angle between two vectors, or equivalently the dot product between their normalizations. A popular application is to quantify semantic similarity between high-dimensional objects by applying cosine-similarity to a learned low-dimensional feature embedding. This can work better but sometimes also worse than the unnormalized dot-product between embedded vectors in practice. To gain insight into this empirical observation, we study embeddings derived from reg…

This https://arxiv.org/abs/2402.16787 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Isoperimetric Profiles and Regular Embeddings of locally compact groups
In this article we extend the notion of $L^p$-measured subgroups couplings; a quantitative asymmetric version of measure equivalence that was introduced by Delabie, Koivisto, Le Maître and Tessera for finitely generated groups, to the setting of locally compact compactly generated unimodular groups. As an example of these couplings; using ideas from Bader and Rosendal, we prove a "dynamical criteria" for the existence of regular embeddings between amenable locally compact compactly generated u…

This https://arxiv.org/abs/2106.14969 has been replaced.
link: https://scholar.google.com/scholar?q=a

Hop-Constrained Metric Embeddings and their Applications
In network design problems, such as compact routing, the goal is to route packets between nodes using the (approximated) shortest paths. A desirable property of these routes is a small number of hops, which makes them more reliable, and reduces the transmission costs. Following the overwhelming success of stochastic tree embeddings for algorithmic design, Haeupler, Hershkowitz, and Zuzic (STOC'21) studied hop-constrained Ramsey-type metric embeddings into trees. Specifically, embedding $f:G(V,E…

Unsupervised Speaker Diarization in Distributed IoT Networks Using Federated Learning
Amit Kumar Bhuyan, Hrishikesh Dutta, Subir Biswas
https://arxiv.org/abs/2404.10842

Unsupervised Speaker Diarization in Distributed IoT Networks Using Federated Learning
This paper presents a computationally efficient and distributed speaker diarization framework for networked IoT-style audio devices. The work proposes a Federated Learning model which can identify the participants in a conversation without the requirement of a large audio database for training. An unsupervised online update mechanism is proposed for the Federated Learning model which depends on cosine similarity of speaker embeddings. Moreover, the proposed diarization system solves the problem…

Simple Techniques for Enhancing Sentence Embeddings in Generative Language Models
Bowen Zhang, Kehua Chang, Chunping Li
https://arxiv.org/abs/2404.03921 ht…

Simple Techniques for Enhancing Sentence Embeddings in Generative Language Models
Sentence Embedding stands as a fundamental task within the realm of Natural Language Processing, finding extensive application in search engines, expert systems, and question-and-answer platforms. With the continuous evolution of large language models such as LLaMA and Mistral, research on sentence embedding has recently achieved notable breakthroughs. However, these advancements mainly pertain to fine-tuning scenarios, leaving explorations into computationally efficient direct inference method…

Is Cosine-Similarity of Embeddings Really About Similarity?
Harald Steck, Chaitanya Ekanadham, Nathan Kallus
https://arxiv.org/abs/2403.05440 https://

Is Cosine-Similarity of Embeddings Really About Similarity?
Cosine-similarity is the cosine of the angle between two vectors, or equivalently the dot product between their normalizations. A popular application is to quantify semantic similarity between high-dimensional objects by applying cosine-similarity to a learned low-dimensional feature embedding. This can work better but sometimes also worse than the unnormalized dot-product between embedded vectors in practice. To gain insight into this empirical observation, we study embeddings derived from reg…

Ellipsoidal embeddings of graphs
Micha\"el Fanuel, Antoine Aspeel, Michael T. Schaub, Jean-Charles Delvenne
https://arxiv.org/abs/2403.15023 https://<…

Ellipsoidal embeddings of graphs
Due to their flexibility to represent almost any kind of relational data, graph-based models have enjoyed a tremendous success over the past decades. While graphs are inherently only combinatorial objects, however, many prominent analysis tools are based on the algebraic representation of graphs via matrices such as the graph Laplacian, or on associated graph embeddings. Such embeddings associate to each node a set of coordinates in a vector space, a representation which can then be employed fo…

This https://arxiv.org/abs/2403.07721 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csHC_…

Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion
How to decode human vision through neural signals has attracted a long-standing interest in neuroscience and machine learning. Modern contrastive learning and generative models improved the performance of fMRI-based visual decoding and reconstruction. However, the high cost and low temporal resolution of fMRI limit their applications in brain-computer interfaces (BCIs), prompting a high need for EEG-based visual reconstruction. In this study, we present an EEG-based visual reconstruction framew…

Unsupervised Speaker Diarization in Distributed IoT Networks Using Federated Learning
Amit Kumar Bhuyan, Hrishikesh Dutta, Subir Biswas
https://arxiv.org/abs/2404.10842

Unsupervised Speaker Diarization in Distributed IoT Networks Using Federated Learning
This paper presents a computationally efficient and distributed speaker diarization framework for networked IoT-style audio devices. The work proposes a Federated Learning model which can identify the participants in a conversation without the requirement of a large audio database for training. An unsupervised online update mechanism is proposed for the Federated Learning model which depends on cosine similarity of speaker embeddings. Moreover, the proposed diarization system solves the problem…

Interpretable User Satisfaction Estimation for Conversational Systems with Large Language Models
Ying-Chun Lin, Jennifer Neville, Jack W. Stokes, Longqi Yang, Tara Safavi, Mengting Wan, Scott Counts, Siddharth Suri, Reid Andersen, Xiaofeng Xu, Deepak Gupta, Sujay Kumar Jauhar, Xia Song, Georg Buscher, Saurabh Tiwary, Brent Hecht, Jaime Teevan
https:/…

Interpretable User Satisfaction Estimation for Conversational Systems with Large Language Models
Accurate and interpretable user satisfaction estimation (USE) is critical for understanding, evaluating, and continuously improving conversational systems. Users express their satisfaction or dissatisfaction with diverse conversational patterns in both general-purpose (ChatGPT and Bing Copilot) and task-oriented (customer service chatbot) conversational systems. Existing approaches based on featurized ML models or text embeddings fall short in extracting generalizable patterns and are hard to i…

This https://arxiv.org/abs/2312.06499 has been replaced.
link: https://scholar.google.com/scholar?q=a

TaCo: Targeted Concept Removal in Output Embeddings for NLP via Information Theory and Explainability
The fairness of Natural Language Processing (NLP) models has emerged as a crucial concern. Information theory indicates that to achieve fairness, a model should not be able to predict sensitive variables, such as gender, ethnicity, and age. However, information related to these variables often appears implicitly in language, posing a challenge in identifying and mitigating biases effectively. To tackle this issue, we present a novel approach that operates at the embedding level of an NLP model,…

Fusion approaches for emotion recognition from speech using acoustic and text-based features
Leonardo Pepino, Pablo Riera, Luciana Ferrer, Agustin Gravano
https://arxiv.org/abs/2403.18635

Fusion approaches for emotion recognition from speech using acoustic and text-based features
In this paper, we study different approaches for classifying emotions from speech using acoustic and text-based features. We propose to obtain contextualized word embeddings with BERT to represent the information contained in speech transcriptions and show that this results in better performance than using Glove embeddings. We also propose and compare different strategies to combine the audio and text modalities, evaluating them on IEMOCAP and MSP-PODCAST datasets. We find that fusing acoustic …

ReliK: A Reliability Measure for Knowledge Graph Embeddings
Maximilian K. Egger, Wenyue Ma, Davide Mottin, Panagiotis Karras, Ilaria Bordino, Francesco Gullo, Aris Anagnostopoulos
https://arxiv.org/abs/2404.16572

ReliK: A Reliability Measure for Knowledge Graph Embeddings
Can we assess a priori how well a knowledge graph embedding will perform on a specific downstream task and in a specific part of the knowledge graph? Knowledge graph embeddings (KGEs) represent entities (e.g., "da Vinci," "Mona Lisa") and relationships (e.g., "painted") of a knowledge graph (KG) as vectors. KGEs are generated by optimizing an embedding score, which assesses whether a triple (e.g., "da Vinci," "painted," "Mona Lisa") exists in the graph. KGEs have been proven effective in a vari…

Automated Similarity Metric Generation for Recommendation
Liang Qu, Yun Lin, Wei Yuan, Xiaojun Wan, Yuhui Shi, Hongzhi Yin
https://arxiv.org/abs/2404.11818

Automated Similarity Metric Generation for Recommendation
The embedding-based architecture has become the dominant approach in modern recommender systems, mapping users and items into a compact vector space. It then employs predefined similarity metrics, such as the inner product, to calculate similarity scores between user and item embeddings, thereby guiding the recommendation of items that align closely with a user's preferences. Given the critical role of similarity metrics in recommender systems, existing methods mainly employ handcrafted similar…

This https://arxiv.org/abs/2307.04384 has been replaced.
link: https://scholar.google.com/scholar?q=a

Neural Causal Graph Collaborative Filtering
Graph collaborative filtering (GCF) has gained considerable attention in recommendation systems by leveraging graph learning techniques to enhance collaborative filtering (CF). One classical approach in GCF is to learn user and item embeddings with Graph Convolutional Network (GCN) and utilize these embeddings for CF models. However, existing GCN-based methods are insufficient in generating satisfactory embeddings for CF models. This is because they fail to model complex node dependencies and v…

This https://arxiv.org/abs/2402.14208 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Content Conditional Debiasing for Fair Text Embedding
Mitigating biases in machine learning models has gained increasing attention in Natural Language Processing (NLP). Yet, only a few studies focus on fair text embeddings, which are crucial yet challenging for real-world applications. In this paper, we propose a novel method for learning fair text embeddings. We achieve fairness while maintaining utility trade-off by ensuring conditional independence between sensitive attributes and text embeddings conditioned on the content. Specifically, we enf…

This https://arxiv.org/abs/2403.15023 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSI_…

Ellipsoidal embeddings of graphs
Due to their flexibility to represent almost any kind of relational data, graph-based models have enjoyed a tremendous success over the past decades. While graphs are inherently only combinatorial objects, however, many prominent analysis tools are based on the algebraic representation of graphs via matrices such as the graph Laplacian, or on associated graph embeddings. Such embeddings associate to each node a set of coordinates in a vector space, a representation which can then be employed fo…

ACORN: Performant and Predicate-Agnostic Search Over Vector Embeddings and Structured Data
Liana Patel, Peter Kraft, Carlos Guestrin, Matei Zaharia
https://arxiv.org/abs/2403.04871

ACORN: Performant and Predicate-Agnostic Search Over Vector Embeddings and Structured Data
Applications increasingly leverage mixed-modality data, and must jointly search over vector data, such as embedded images, text and video, as well as structured data, such as attributes and keywords. Proposed methods for this hybrid search setting either suffer from poor performance or support a severely restricted set of search predicates (e.g., only small sets of equality predicates), making them impractical for many applications. To address this, we present ACORN, an approach for performant …

ACORN: Performant and Predicate-Agnostic Search Over Vector Embeddings and Structured Data
Liana Patel, Peter Kraft, Carlos Guestrin, Matei Zaharia
https://arxiv.org/abs/2403.04871

ACORN: Performant and Predicate-Agnostic Search Over Vector Embeddings and Structured Data
Applications increasingly leverage mixed-modality data, and must jointly search over vector data, such as embedded images, text and video, as well as structured data, such as attributes and keywords. Proposed methods for this hybrid search setting either suffer from poor performance or support a severely restricted set of search predicates (e.g., only small sets of equality predicates), making them impractical for many applications. To address this, we present ACORN, an approach for performant …

This https://arxiv.org/abs/2305.12517 has been replaced.
link: https://scholar.google.com/scholar?q=a

Description-Based Text Similarity
Identifying texts with a given semantics is central for many information seeking scenarios. Similarity search over vector embeddings appear to be central to this ability, yet the similarity reflected in current text embeddings is corpus-driven, and is inconsistent and sub-optimal for many use cases. What, then, is a good notion of similarity for effective retrieval of text? We identify the need to search for texts based on abstract descriptions of their content, and the corresponding notion o…

This https://arxiv.org/abs/2403.18479 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIR_…

Lightweight Embeddings for Graph Collaborative Filtering
Graph neural networks (GNNs) are currently one of the most performant collaborative filtering methods. Meanwhile, owing to the use of an embedding table to represent each user/item as a distinct vector, GNN-based recommenders have inherited the long-standing defect of parameter inefficiency. As a common practice for scalable embeddings, parameter sharing enables the use of fewer embedding vectors (i.e., meta-embeddings). When assigning meta-embeddings, most existing methods are a heuristically …

This https://arxiv.org/abs/2309.05494 has been replaced.
link: https://scholar.google.com/scholar?q=a

CrisisTransformers: Pre-trained language models and sentence encoders for crisis-related social media texts
Social media platforms play an essential role in crisis communication, but analyzing crisis-related social media texts is challenging due to their informal nature. Transformer-based pre-trained models like BERT and RoBERTa have shown success in various NLP tasks, but they are not tailored for crisis-related texts. Furthermore, general-purpose sentence encoders are used to generate sentence embeddings, regardless of the textual complexities in crisis-related texts. Advances in applications like …

Meta-Task Prompting Elicits Embedding from Large Language Models
Yibin Lei, Di Wu, Tianyi Zhou, Tao Shen, Yu Cao, Chongyang Tao, Andrew Yates
https://arxiv.org/abs/2402.18458

Meta-Task Prompting Elicits Embedding from Large Language Models
In this work, we introduce a new unsupervised embedding method, Meta-Task Prompting with Explicit One-Word Limitation (MetaEOL), for generating high-quality sentence embeddings from Large Language Models (LLMs) without the need for model fine-tuning or task-specific engineering. Leveraging meta-task prompting, MetaEOL guides LLMs to produce embeddings through a series of carefully designed prompts that address multiple representational aspects. Our comprehensive experiments demonstrate that emb…

This https://arxiv.org/abs/2310.07815 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIR_…

Language Models As Semantic Indexers
Semantic identifier (ID) is an important concept in information retrieval that aims to preserve the semantics of objects such as documents and items inside their IDs. Previous studies typically adopt a two-stage pipeline to learn semantic IDs by first procuring embeddings using off-the-shelf text encoders and then deriving IDs based on the embeddings. However, each step introduces potential information loss, and there is usually an inherent mismatch between the distribution of embeddings within…

This https://arxiv.org/abs/2305.12517 has been replaced.
link: https://scholar.google.com/scholar?q=a

Description-Based Text Similarity
Identifying texts with a given semantics is central for many information seeking scenarios. Similarity search over vector embeddings appear to be central to this ability, yet the similarity reflected in current text embeddings is corpus-driven, and is inconsistent and sub-optimal for many use cases. What, then, is a good notion of similarity for effective retrieval of text? We identify the need to search for texts based on abstract descriptions of their content, and the corresponding notion o…

This https://arxiv.org/abs/2405.03562 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIR_…

ID-centric Pre-training for Recommendation
Classical sequential recommendation models generally adopt ID embeddings to store knowledge learned from user historical behaviors and represent items. However, these unique IDs are challenging to be transferred to new domains. With the thriving of pre-trained language model (PLM), some pioneer works adopt PLM for pre-trained recommendation, where modality information (e.g., text) is considered universal across domains via PLM. Unfortunately, the behavioral information in ID embeddings is still…

DGR: A General Graph Desmoothing Framework for Recommendation via Global and Local Perspectives
Leilei Ding, Dazhong Shen, Chao Wang, Tianfu Wang, Le Zhang, Hui Xiong, Yanyong Zhang
https://arxiv.org/abs/2403.04287

DGR: A General Graph Desmoothing Framework for Recommendation via Global and Local Perspectives
Graph Convolutional Networks (GCNs) have become pivotal in recommendation systems for learning user and item embeddings by leveraging the user-item interaction graph's node information and topology. However, these models often face the famous over-smoothing issue, leading to indistinct user and item embeddings and reduced personalization. Traditional desmoothing methods in GCN-based systems are model-specific, lacking a universal solution. This paper introduces a novel, model-agnostic approach …

ID-centric Pre-training for Recommendation
Yiqing Wu, Ruobing Xie, Zhao Zhang, Fuzhen Zhuang, Xu Zhang, Leyu Lin, Zhanhui Kang, Yongjun Xu
https://arxiv.org/abs/2405.03562

ID-centric Pre-training for Recommendation
Classical sequential recommendation models generally adopt ID embeddings to store knowledge learned from user historical behaviors and represent items. However, these unique IDs are challenging to be transferred to new domains. With the thriving of pre-trained language model (PLM), some pioneer works adopt PLM for pre-trained recommendation, where modality information (e.g., text) is considered universal across domains via PLM. Unfortunately, the behavioral information in ID embeddings is still…

This https://arxiv.org/abs/2403.17416 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIR_…

AFDGCF: Adaptive Feature De-correlation Graph Collaborative Filtering for Recommendations
Collaborative filtering methods based on graph neural networks (GNNs) have witnessed significant success in recommender systems (RS), capitalizing on their ability to capture collaborative signals within intricate user-item relationships via message-passing mechanisms. However, these GNN-based RS inadvertently introduce excess linear correlation between user and item embeddings, contradicting the goal of providing personalized recommendations. While existing research predominantly ascribes this…

OmniSearchSage: Multi-Task Multi-Entity Embeddings for Pinterest Search
Prabhat Agarwal, Minhazul Islam Sk, Nikil Pancha, Kurchi Subhra Hazra, Jiajing Xu, Chuck Rosenberg
https://arxiv.org/abs/2404.16260

OmniSearchSage: Multi-Task Multi-Entity Embeddings for Pinterest Search
In this paper, we present OmniSearchSage, a versatile and scalable system for understanding search queries, pins, and products for Pinterest search. We jointly learn a unified query embedding coupled with pin and product embeddings, leading to an improvement of $>8\%$ relevance, $>7\%$ engagement, and $>5\%$ ads CTR in Pinterest's production search system. The main contributors to these gains are improved content understanding, better multi-task learning, and real-time serving. We enrich our en…

This https://arxiv.org/abs/2402.16073 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIR_…

Pfeed: Generating near real-time personalized feeds using precomputed embedding similarities
In personalized recommender systems, embeddings are often used to encode customer actions and items, and retrieval is then performed in the embedding space using approximate nearest neighbor search. However, this approach can lead to two challenges: 1) user embeddings can restrict the diversity of interests captured and 2) the need to keep them up-to-date requires an expensive, real-time infrastructure. In this paper, we propose a method that overcomes these challenges in a practical, industria…

Sign-Guided Bipartite Graph Hashing for Hamming Space Search
Xueyi Wu
https://arxiv.org/abs/2405.02716 https://arxiv.org/pdf/2405.027…

Sign-Guided Bipartite Graph Hashing for Hamming Space Search
Bipartite graph hashing (BGH) is extensively used for Top-K search in Hamming space at low storage and inference costs. Recent research adopts graph convolutional hashing for BGH and has achieved the state-of-the-art performance. However, the contributions of its various influencing factors to hashing performance have not been explored in-depth, including the same/different sign count between two binary embeddings during Hamming space search (sign property), the contribution of sub-embeddings a…