Tootfinder

Analyzing the Role of Semantic Representations in the Era of Large Language Models
Zhijing Jin, Yuen Chen, Fernando Gonzalez, Jiarui Liu, Jiayi Zhang, Julian Michael, Bernhard Sch\"olkopf, Mona Diab
https://arxiv.org/abs/2405.01502

Analyzing the Role of Semantic Representations in the Era of Large Language Models
Traditionally, natural language processing (NLP) models often use a rich set of features created by linguistic expertise, such as semantic representations. However, in the era of large language models (LLMs), more and more tasks are turned into generic, end-to-end sequence generation problems. In this paper, we investigate the question: what is the role of semantic representations in the era of LLMs? Specifically, we investigate the effect of Abstract Meaning Representation (AMR) across five di…

A Survey on Semantic Communication Networks: Architecture, Security, and Privacy
Shaolong Guo, Yuntao Wang, Ning Zhang, Zhou Su, Tom H. Luan, Zhiyi Tian, Xuemin Shen
https://arxiv.org/abs/2405.01221

A Survey on Semantic Communication Networks: Architecture, Security, and Privacy
Semantic communication, emerging as a breakthrough beyond the classical Shannon paradigm, aims to convey the essential meaning of source data rather than merely focusing on precise yet content-agnostic bit transmission. By interconnecting diverse intelligent agents (e.g., autonomous vehicles and VR devices) via semantic communications, the semantic communication networks (SemComNet) supports semantic-oriented transmission, efficient spectrum utilization, and flexible networking among collaborat…

Reasoning About Group Polarization: From Semantic Games to Sequent Systems
Robert Freiman, Carlos Olarte, Elaine Pimentel, Christian G. Ferm\"uller
https://arxiv.org/abs/2405.01322

Reasoning About Group Polarization: From Semantic Games to Sequent Systems
Group polarization, the phenomenon where individuals become more extreme after interacting, has been gaining attention, especially with the rise of social media shaping people's opinions. Recent interest has emerged in formal reasoning about group polarization using logical systems. In this work we consider the modal logic PNL that captures the notion of agents agreeing or disagreeing on a given topic. Our contribution involves enhancing PNL with advanced formal reasoning techniques, instead of…

This https://arxiv.org/abs/2310.07815 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIR_…

Language Models As Semantic Indexers
Semantic identifier (ID) is an important concept in information retrieval that aims to preserve the semantics of objects such as documents and items inside their IDs. Previous studies typically adopt a two-stage pipeline to learn semantic IDs by first procuring embeddings using off-the-shelf text encoders and then deriving IDs based on the embeddings. However, each step introduces potential information loss, and there is usually an inherent mismatch between the distribution of embeddings within…

Hybrid Bit and Semantic Communications
Kaiwen Yu, Renhe Fan, Gang Wu, Zhijin Qin
https://arxiv.org/abs/2404.19477 https://arxiv.org/p…

Hybrid Bit and Semantic Communications
Semantic communication technology is regarded as a method surpassing the Shannon limit of bit transmission, capable of effectively enhancing transmission efficiency. However, current approaches that directly map content to transmission symbols are challenging to deploy in practice, imposing significant limitations on the development of semantic communication. To address this challenge, we propose a hybrid bit and semantic communication system, named HybridBSC, in which encoded semantic informat…

SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound
Haohe Liu, Xuenan Xu, Yi Yuan, Mengyue Wu, Wenwu Wang, Mark D. Plumbley
https://arxiv.org/abs/2405.00233 …

SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound
Large language models (LLMs) have significantly advanced audio processing through audio codecs that convert audio into discrete tokens, enabling the application of language modelling techniques to audio data. However, traditional codecs often operate at high bitrates or within narrow domains such as speech and lack the semantic clues required for efficient language modelling. Addressing these challenges, we introduce SemantiCodec, a novel codec designed to compress audio into fewer than a hundr…

A Joint Communication and Computation Design for Distributed RISs Assisted Probabilistic Semantic Communication in IIoT
Zhouxiang Zhao, Zhaohui Yang, Chongwen Huang, Li Wei, Qianqian Yang, Caijun Zhong, Wei Xu, Zhaoyang Zhang
https://arxiv.org/abs/2404.19750

A Joint Communication and Computation Design for Distributed RISs Assisted Probabilistic Semantic Communication in IIoT
In this paper, the problem of spectral-efficient communication and computation resource allocation for distributed reconfigurable intelligent surfaces (RISs) assisted probabilistic semantic communication (PSC) in industrial Internet-of-Things (IIoT) is investigated. In the considered model, multiple RISs are deployed to serve multiple users, while PSC adopts compute-then-transmit protocol to reduce the transmission data size. To support high-rate transmission, the semantic compression ratio, tr…

ReeSPOT: Reeb Graph Models Semantic Patterns of Normalcy in Human Trajectories
Bowen Zhang, Shailja S., Chandrakanth Gudavalli, Connor Levenson, Amil Khan, B. S. Manjunath
https://arxiv.org/abs/2405.00808

ReeSPOT: Reeb Graph Models Semantic Patterns of Normalcy in Human Trajectories
This paper introduces ReeSPOT, a novel Reeb graph-based method to model patterns of life in human trajectories (akin to a fingerprint). Human behavior typically follows a pattern of normalcy in day-to-day activities. This is marked by recurring activities within specific time periods. In this paper, we model this behavior using Reeb graphs where any deviation from usual day-to-day activities is encoded as nodes in the Reeb graph. The complexity of the proposed algorithm is linear with respect t…

This https://arxiv.org/abs/2309.10924 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csRO_…

Change of Scenery: Unsupervised LiDAR Change Detection for Mobile Robots
This paper presents a fully unsupervised deep change detection approach for mobile robots with 3D LiDAR. In unstructured environments, it is infeasible to define a closed set of semantic classes. Instead, semantic segmentation is reformulated as binary change detection. We develop a neural network, RangeNetCD, that uses an existing point-cloud map and a live LiDAR scan to detect scene changes with respect to the map. Using a novel loss function, existing point-cloud semantic segmentation networ…

Over the last few weeks, I’ve been a really big fan of PicoCSS. Drop a #CSS file in an #HTML file, write some semantic HTML, and it looks nice.
https://picocs…

Pico CSS • Minimal CSS Framework for semantic HTML
Minimalist and lightweight starter kit that prioritizes semantic syntax, making every HTML element responsive and elegant by default.

New paper: Semantic differences in visually similar face emojis 📝
We show that even #emojis that look similar (like 😊, ☺️) have subtle, reliable meaning differences, supporting a lexicalist approach. #linguistics

Presenting Profunctors
Gabriel Goren Roig, Joshua Meyers, Emilio Minichiello
https://arxiv.org/abs/2404.01406 https://arxiv.org/pdf/2…

Presenting Profunctors
Motivated by problems in categorical database theory, we introduce and compare two notions of presentation for profunctors, uncurried and curried, which arise intuitively from thinking of profunctors either as functors $\mathcal{C}^\text{op} \times \mathcal{D} \to \textbf{Set}$ or $\mathcal{C}^\text{op} \to \textbf{Set}^{\mathcal{D}}$. Although the Cartesian closure of $\textbf{Cat}$ means these two perspectives can be used interchangeably at the semantic level, a surprising amount of subtlety …

This https://arxiv.org/abs/2404.14749 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

Semantic Cells: Evolutional Process to Acquire Sense Diversity of Items
Previous models for learning the semantic vectors of items and their groups, such as words, sentences, nodes, and graphs, using distributed representation have been based on the assumption that an item corresponds to one vector composed of dimensions corresponding to hidden contexts in the target. Multiple senses of an item are represented by assigning a vector to each of the domains where the item may appear or reflecting the context to the sense of the item. However, there may be multiple dis…

SeD: Semantic-Aware Discriminator for Image Super-Resolution
Bingchen Li, Xin Li, Hanxin Zhu, Yeying Jin, Ruoyu Feng, Zhizheng Zhang, Zhibo Chen
https://arxiv.org/abs/2402.19387 <…

SeD: Semantic-Aware Discriminator for Image Super-Resolution
Generative Adversarial Networks (GANs) have been widely used to recover vivid textures in image super-resolution (SR) tasks. In particular, one discriminator is utilized to enable the SR network to learn the distribution of real-world high-quality images in an adversarial training manner. However, the distribution learning is overly coarse-grained, which is susceptible to virtual textures and causes counter-intuitive generation results. To mitigate this, we propose the simple and effective Sema…

Characterising the Creative Process in Humans and Large Language Models
Surabhi S. Nath, Peter Dayan, Claire Stevenson
https://arxiv.org/abs/2405.00899 htt…

Characterising the Creative Process in Humans and Large Language Models
Large language models appear quite creative, often performing on par with the average human on creative tasks. However, research on LLM creativity has focused solely on \textit{products}, with little attention on the creative \textit{process}. Process analyses of human creativity often require hand-coded categories or exploit response times, which do not apply to LLMs. We provide an automated method to characterise how humans and LLMs explore semantic spaces on the Alternate Uses Task, and cont…

ReeSPOT: Reeb Graph Models Semantic Patterns of Normalcy in Human Trajectories
Bowen Zhang, Shailja S., Chandrakanth Gudavalli, Connor Levenson, Amil Khan, B. S. Manjunath
https://arxiv.org/abs/2405.00808

ReeSPOT: Reeb Graph Models Semantic Patterns of Normalcy in Human Trajectories
This paper introduces ReeSPOT, a novel Reeb graph-based method to model patterns of life in human trajectories (akin to a fingerprint). Human behavior typically follows a pattern of normalcy in day-to-day activities. This is marked by recurring activities within specific time periods. In this paper, we model this behavior using Reeb graphs where any deviation from usual day-to-day activities is encoded as nodes in the Reeb graph. The complexity of the proposed algorithm is linear with respect t…

Generation of Asset Administration Shell with Large Language Model Agents: Interoperability in Digital Twins with Semantic Node
Yuchen Xia, Zhewen Xiao, Nasser Jazdi, Michael Weyrich
https://arxiv.org/abs/2403.17209

Generation of Asset Administration Shell with Large Language Model Agents: Interoperability in Digital Twins with Semantic Node
This research introduces a novel approach for assisting the creation of Asset Administration Shell (AAS) instances for digital twin modeling within the context of Industry 4.0, aiming to enhance interoperability in smart manufacturing and reduce manual effort. We construct a "semantic node" data structure to capture the semantic essence of textual data. Then, a system powered by large language models is designed and implemented to process "semantic node" and generate AAS instance models from te…

Evolving Semantic Communication with Generative Model
Shunpu Tang, Qianqian Yang, Deniz G\"und\"uz, Zhaoyang Zhang
https://arxiv.org/abs/2403.20237

Evolving Semantic Communication with Generative Model
Recently, learning-based semantic communication (SemCom) has emerged as a promising approach in the upcoming 6G network and researchers have made remarkable efforts in this field. However, existing works have yet to fully explore the advantages of the evolving nature of learning-based systems, where knowledge accumulates during transmission have the potential to enhance system performance. In this paper, we explore an evolving semantic communication system for image transmission, referred to as…

Semantic Image Synthesis with Unconditional Generator
Jungwoo Chae, Hyunin Cho, Sooyeon Go, Kyungmook Choi, Youngjung Uh
https://arxiv.org/abs/2402.14395 h…

Semantic Image Synthesis with Unconditional Generator
Semantic image synthesis (SIS) aims to generate realistic images that match given semantic masks. Despite recent advances allowing high-quality results and precise spatial control, they require a massive semantic segmentation dataset for training the models. Instead, we propose to employ a pre-trained unconditional generator and rearrange its feature maps according to proxy masks. The proxy masks are prepared from the feature maps of random samples in the generator by simple clustering. The fea…

This https://arxiv.org/abs/2404.13696 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csRO_…

Clio: Real-time Task-Driven Open-Set 3D Scene Graphs
Modern tools for class-agnostic image segmentation (e.g., SegmentAnything) and open-set semantic understanding (e.g., CLIP) provide unprecedented opportunities for robot perception and mapping. While traditional closed-set metric-semantic maps were restricted to tens or hundreds of semantic classes, we can now build maps with a plethora of objects and countless semantic variations. This leaves us with a fundamental question: what is the right granularity for the objects (and, more generally, fo…

This https://arxiv.org/abs/2310.07576 has been replaced.
link: https://scholar.google.com/scholar?q=a

Analyzing Trendy Twitter Hashtags in the 2022 French Election
Regressions trained to predict the future activity of social media users need rich features for accurate predictions. Many advanced models exist to generate such features; however, the time complexities of their computations are often prohibitive when they run on enormous data-sets. Some studies have shown that simple semantic network features can be rich enough to use for regressions without requiring complex computations. We propose a method for using semantic networks as user-level features …

S\~onajaht: Definition Embeddings and Semantic Search for Reverse Dictionary Creation
Aleksei Dorkin, Kairit Sirts
https://arxiv.org/abs/2404.19430 https://arxiv.org/pdf/2404.19430
arXiv:2404.19430v1 Announce Type: new
Abstract: We present an information retrieval based reverse dictionary system using modern pre-trained language models and approximate nearest neighbors search algorithms. The proposed approach is applied to an existing Estonian language lexicon resource, S\~onaveeb (word web), with the purpose of enhancing and enriching it by introducing cross-lingual reverse dictionary functionality powered by semantic search.
The performance of the system is evaluated using both an existing labeled English dataset of words and definitions that is extended to contain also Estonian and Russian translations, and a novel unlabeled evaluation approach that extracts the evaluation data from the lexicon resource itself using synonymy relations.
Evaluation results indicate that the information retrieval based semantic search approach without any model training is feasible, producing median rank of 1 in the monolingual setting and median rank of 2 in the cross-lingual setting using the unlabeled evaluation approach, with models trained for cross-lingual retrieval and including Estonian in their training data showing superior performance in our particular task.

Equational Theories and Validity for Logically Constrained Term Rewriting (Full Version)
Takahito Aoto, Naoki Nishida, Jonas Sch\"opf
https://arxiv.org/abs/2405.01174

Equational Theories and Validity for Logically Constrained Term Rewriting (Full Version)
Logically constrained term rewriting is a relatively new formalism where rules are equipped with constraints over some arbitrary theory. Although there are many recent advances with respect to rewriting induction, completion, complexity analysis and confluence analysis for logically constrained term rewriting, these works solely focus on the syntactic side of the formalism lacking detailed investigations on semantics. In this paper, we investigate a semantic side of logically constrained term r…

At this year's Berlin Buzzwords, join Kentaro Takiguchi for his talk on integrating semantic search into an established lexical search system, addressing potential challenges and pitfalls, and evaluating different optimisation methods and their varying effects on metrics by exploring and enhancing lexical and semantic search in practical scenarios. #bbuzz

Presenting Profunctors
Gabriel Goren Roig, Joshua Meyers, Emilio Minichiello
https://arxiv.org/abs/2404.01406 https://arxiv.org/pdf/2…

Presenting Profunctors
Motivated by problems in categorical database theory, we introduce and compare two notions of presentation for profunctors, uncurried and curried, which arise intuitively from thinking of profunctors either as functors $\mathcal{C}^\text{op} \times \mathcal{D} \to \textbf{Set}$ or $\mathcal{C}^\text{op} \to \textbf{Set}^{\mathcal{D}}$. Although the Cartesian closure of $\textbf{Cat}$ means these two perspectives can be used interchangeably at the semantic level, a surprising amount of subtlety …

This https://arxiv.org/abs/2310.06356 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCR_…

A Semantic Invariant Robust Watermark for Large Language Models
Watermark algorithms for large language models (LLMs) have achieved extremely high accuracy in detecting text generated by LLMs. Such algorithms typically involve adding extra watermark logits to the LLM's logits at each generation step. However, prior algorithms face a trade-off between attack robustness and security robustness. This is because the watermark logits for a token are determined by a certain number of preceding tokens; a small number leads to low security robustness, while a large…

Codebook-enabled Generative End-to-end Semantic Communication Powered by Transformer
Peigen Ye, Yaping Sun, Shumin Yao, Hao Chen, Xiaodong Xu, Shuguang Cui
https://arxiv.org/abs/2402.16868

Codebook-enabled Generative End-to-end Semantic Communication Powered by Transformer
Codebook-based generative semantic communication attracts increasing attention, since only indices are required to be transmitted when the codebook is shared between transmitter and receiver. However, due to the fact that the semantic relations among code vectors are not necessarily related to the distance of the corresponding code indices, the performance of the codebook-enabled semantic communication system is susceptible to the channel noise. Thus, how to improve the system robustness agains…

Qualia and the Formal Structure of Meaning
Xerxes D. Arsiwalla
https://arxiv.org/abs/2405.01148 https://arxiv.org/pdf/2405.01148

Qualia and the Formal Structure of Meaning
This work explores the hypothesis that subjectively attributed meaning constitutes the phenomenal content of conscious experience. That is, phenomenal content is semantic. This form of subjective meaning manifests as an intrinsic and non-representational character of qualia. Empirically, subjective meaning is ubiquitous in conscious experiences. We point to phenomenological studies that lend evidence to support this. Furthermore, this notion of meaning closely relates to what Frege refers to as…

Without intentional synthesis, "data driven" always means validating the biases baked into your semantic environment.
Data points that don't fit an org's established assumptions are easily swept under the rug.
Anyone interested in real insights needs to go data gathering and adopt a synthesis process that allows them to holistically inhabit that data, and intentionally curate a receptive semantic environment.

Don’t make “data-driven product decisions” — build a data-driven semantic environment
Like “growth mindset” or “agile development,” data-driven decision making has snuck its way into the way almost every product org describes itself. The trouble is that it doesn’t mean anything —…

Synthetic Data for Robust Stroke Segmentation
Liam Chalcroft, Ioannis Pappas, Cathy J. Price, John Ashburner
https://arxiv.org/abs/2404.01946 https://

Synthetic Data for Robust Stroke Segmentation
Deep learning-based semantic segmentation in neuroimaging currently requires high-resolution scans and extensive annotated datasets, posing significant barriers to clinical applicability. We present a novel synthetic framework for the task of lesion segmentation, extending the capabilities of the established SynthSeg approach to accommodate large heterogeneous pathologies with lesion-specific augmentation strategies. Our method trains deep learning models, demonstrated here with the UNet archit…

Exploring Weighted Property Approaches for RDF Graph Similarity Measure
Ngoc Luyen Le, Marie-H\'el\`ene Abel, Philippe Gouspillou
https://arxiv.org/abs/2404.19052

Exploring Weighted Property Approaches for RDF Graph Similarity Measure
Measuring similarity between RDF graphs is essential for various applications, including knowledge discovery, semantic web analysis, and recommender systems. However, traditional similarity measures often treat all properties equally, potentially overlooking the varying importance of different properties in different contexts. Consequently, exploring weighted property approaches for RDF graph similarity measure presents an intriguing avenue for investigation. Therefore, in this paper, we propos…

This https://arxiv.org/abs/2311.11016 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csRO_…

SNI-SLAM: Semantic Neural Implicit SLAM
We propose SNI-SLAM, a semantic SLAM system utilizing neural implicit representation, that simultaneously performs accurate semantic mapping, high-quality surface reconstruction, and robust camera tracking. In this system, we introduce hierarchical semantic representation to allow multi-level semantic comprehension for top-down structured semantic mapping of the scene. In addition, to fully utilize the correlation between multiple attributes of the environment, we integrate appearance, geometry…

KGUF: Simple Knowledge-aware Graph-based Recommender with User-based Semantic Features Filtering
Salvatore Bufi, Alberto Carlo Maria Mancino, Antonio Ferrara, Daniele Malitesta, Tommaso Di Noia, Eugenio Di Sciascio
https://arxiv.org/abs/2403.20095

KGUF: Simple Knowledge-aware Graph-based Recommender with User-based Semantic Features Filtering
The recent integration of Graph Neural Networks (GNNs) into recommendation has led to a novel family of Collaborative Filtering (CF) approaches, namely Graph Collaborative Filtering (GCF). Following the same GNNs wave, recommender systems exploiting Knowledge Graphs (KGs) have also been successfully empowered by the GCF rationale to combine the representational power of GNNs with the semantics conveyed by KGs, giving rise to Knowledge-aware Graph Collaborative Filtering (KGCF), which use KGs to…

Understanding Archives: Towards New Research Interfaces Relying on the Semantic Annotation of Documents
Nicolas Gutehrl\'e (CRIT), Iana Atanassova (CRIT, STIH, TESNIERE, LaLIC)
https://arxiv.org/abs/2403.19201

Understanding Archives: Towards New Research Interfaces Relying on the Semantic Annotation of Documents
The digitisation campaigns carried out by libraries and archives in recent years have facilitated access to documents in their collections. However, exploring and exploiting these documents remain difficult tasks due to the sheer quantity of documents available for consultation. In this article, we show how the semantic annotation of the textual content of study corpora of archival documents allow to facilitate their exploitation and valorisation. First, we present a methodological framework fo…

This https://arxiv.org/abs/2107.11190 has been replaced.
link: https://scholar.google.com/scholar?q=a

Semantic Communications for Speech Recognition
The traditional communications transmit all the source data represented by bits, regardless of the content of source and the semantic information required by the receiver. However, in some applications, the receiver only needs part of the source data that represents critical semantic information, which prompts to transmit the application-related information, especially when bandwidth resources are limited. In this paper, we consider a semantic communication system for speech recognition by desi…

This https://arxiv.org/abs/2403.11542 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Topology Data Analysis-based Error Detection for Semantic Image Transmission with Incremental Knowledge-based HARQ
Semantic communication (SemCom) aims to achieve high fidelity information delivery under low communication consumption by only guaranteeing semantic accuracy. Nevertheless, semantic communication still suffers from unexpected channel volatility and thus developing a re-transmission mechanism (e.g., hybrid automatic repeat request [HARQ]) is indispensable. In that regard, instead of discarding previously transmitted information, the incremental knowledge-based HARQ (IK-HARQ) is deemed as a more …

Unsupervised Flow Discovery from Task-oriented Dialogues
Patr\'icia Ferreira, Daniel Martins, Ana Alves, Catarina Silva, Hugo Gon\c{c}alo Oliveira
https://arxiv.org/abs/2405.01403

Unsupervised Flow Discovery from Task-oriented Dialogues
The design of dialogue flows is a critical but time-consuming task when developing task-oriented dialogue (TOD) systems. We propose an approach for the unsupervised discovery of flows from dialogue history, thus making the process applicable to any domain for which such an history is available. Briefly, utterances are represented in a vector space and clustered according to their semantic similarity. Clusters, which can be seen as dialogue states, are then used as the vertices of a transition g…

Surgment: Segmentation-enabled Semantic Search and Creation of Visual Question and Feedback to Support Video-Based Surgery Learning
Jingying Wang, Haoran Tang, Taylor Kantor, Tandis Soltani, Vitaliy Popov, Xu Wang
https://arxiv.org/abs/2402.17903

Surgment: Segmentation-enabled Semantic Search and Creation of Visual Question and Feedback to Support Video-Based Surgery Learning
Videos are prominent learning materials to prepare surgical trainees before they enter the operating room (OR). In this work, we explore techniques to enrich the video-based surgery learning experience. We propose Surgment, a system that helps expert surgeons create exercises with feedback based on surgery recordings. Surgment is powered by a few-shot-learning-based pipeline (SegGPT+SAM) to segment surgery scenes, achieving an accuracy of 92\%. The segmentation pipeline enables functionalities …

This https://arxiv.org/abs/2205.02422 has been replaced.
link: https://scholar.google.com/scholar?q=a

Quantum Semantic Communications for Resource-Efficient Quantum Networking
Quantum communication networks (QCNs) utilize quantum mechanics for secure information transmission, but the reliance on fragile and expensive photonic quantum resources renders QCN resource optimization challenging. Unlike prior QCN works that relied on blindly compressing direct quantum embeddings of classical data, this letter proposes a novel quantum semantic communications (QSC) framework exploiting advancements in quantum machine learning and quantum semantic representations to extracts a…

This https://arxiv.org/abs/2404.16573 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCV_…

Multi-Scale Representations by Varying Window Attention for Semantic Segmentation
Multi-scale learning is central to semantic segmentation. We visualize the effective receptive field (ERF) of canonical multi-scale representations and point out two risks in learning them: scale inadequacy and field inactivation. A novel multi-scale learner, varying window attention (VWA), is presented to address these issues. VWA leverages the local window attention (LWA) and disentangles LWA into the query window and context window, allowing the context's scale to vary for the query to learn…

This https://arxiv.org/abs/2212.01485 has been replaced.
link: https://scholar.google.com/scholar?q=a

A Theory of Semantic Communication
Semantic communication is an emerging research area that has gained a wide range of attention recently. Despite this growing interest, there remains a notable absence of a comprehensive and widely-accepted framework for characterizing semantic communication. This paper introduces a new conceptualization of semantic communication and formulates two fundamental problems, which we term language exploitation and language design. Our contention is that the challenge of language design can be effecti…

Multi-hop Question Answering over Knowledge Graphs using Large Language Models
Abir Chakraborty
https://arxiv.org/abs/2404.19234 https://

Multi-hop Question Answering over Knowledge Graphs using Large Language Models
Knowledge graphs (KGs) are large datasets with specific structures representing large knowledge bases (KB) where each node represents a key entity and relations amongst them are typed edges. Natural language queries formed to extract information from a KB entail starting from specific nodes and reasoning over multiple edges of the corresponding KG to arrive at the correct set of answer nodes. Traditional approaches of question answering on KG are based on (a) semantic parsing (SP), where a logi…

A BiRGAT Model for Multi-intent Spoken Language Understanding with Hierarchical Semantic Frames
Hongshen Xu, Ruisheng Cao, Su Zhu, Sheng Jiang, Hanchong Zhang, Lu Chen, Kai Yu
https://arxiv.org/abs/2402.18258

A BiRGAT Model for Multi-intent Spoken Language Understanding with Hierarchical Semantic Frames
Previous work on spoken language understanding (SLU) mainly focuses on single-intent settings, where each input utterance merely contains one user intent. This configuration significantly limits the surface form of user utterances and the capacity of output semantics. In this work, we first propose a Multi-Intent dataset which is collected from a realistic in-Vehicle dialogue System, called MIVS. The target semantic frame is organized in a 3-layer hierarchical structure to tackle the alignment …

This https://arxiv.org/abs/2310.12931 has been replaced.
link: https://scholar.google.com/scholar?q=a

Eureka: Human-Level Reward Design via Coding Large Language Models
Large Language Models (LLMs) have excelled as high-level semantic planners for sequential decision-making tasks. However, harnessing them to learn complex low-level manipulation tasks, such as dexterous pen spinning, remains an open problem. We bridge this fundamental gap and present Eureka, a human-level reward design algorithm powered by LLMs. Eureka exploits the remarkable zero-shot generation, code-writing, and in-context improvement capabilities of state-of-the-art LLMs, such as GPT-4, to …

Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language Model
Yu Cui, Feng Liu, Pengbo Wang, Bohao Wang, Heng Tang, Yi Wan, Jun Wang, Jiawei Chen
https://arxiv.org/abs/2405.00338

Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language Model
Owing to their powerful semantic reasoning capabilities, Large Language Models (LLMs) have been effectively utilized as recommenders, achieving impressive performance. However, the high inference latency of LLMs significantly restricts their practical deployment. To address this issue, this work investigates knowledge distillation from cumbersome LLM-based recommendation models to lightweight conventional sequential models. It encounters three challenges: 1) the teacher's knowledge may not alwa…

Semantic Communication-Enabled Wireless Adaptive Panoramic Video Transmission
Haixiao Gao, Mengying Sun, Xiaodong Xu, Shujun Han
https://arxiv.org/abs/2402.16581

Semantic Communication-Enabled Wireless Adaptive Panoramic Video Transmission
In this paper, we propose an adaptive panoramic video semantic transmission (APVST) network built on the deep joint source-channel coding (Deep JSCC) structure for the efficient end-to-end transmission of panoramic videos. The proposed APVST network can adaptively extract semantic features of panoramic frames and achieve semantic feature encoding. To achieve high spectral efficiency and save bandwidth, we propose a transmission rate control mechanism for the APVST via the entropy model and the …

This https://arxiv.org/abs/2402.09871 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSD_…

MuChin: A Chinese Colloquial Description Benchmark for Evaluating Language Models in the Field of Music
The rapidly evolving multimodal Large Language Models (LLMs) urgently require new benchmarks to uniformly evaluate their performance on understanding and textually describing music. However, due to semantic gaps between Music Information Retrieval (MIR) algorithms and human understanding, discrepancies between professionals and the public, and low precision of annotations, existing music description datasets cannot serve as benchmarks. To this end, we present MuChin, the first open-source music…

This https://arxiv.org/abs/2203.05314 has been replaced.
link: https://scholar.google.com/scholar?q=a

SoK: On the Semantic AI Security in Autonomous Driving
Autonomous Driving (AD) systems rely on AI components to make safety and correct driving decisions. Unfortunately, today's AI algorithms are known to be generally vulnerable to adversarial attacks. However, for such AI component-level vulnerabilities to be semantically impactful at the system level, it needs to address non-trivial semantic gaps both (1) from the system-level attack input spaces to those at AI component level, and (2) from AI component-level attack impacts to those at the system…

Semantic Cells: Evolutional Process to Acquire Sense Diversity of Items
Yukio Ohsawa, Dingding Xu, Kaira Sekiguchi
https://arxiv.org/abs/2404.14749 https:/…

Semantic Cells: Evolutional Process to Acquire Sense Diversity of Items
Previous models for learning the semantic vectors of items and their groups, such as words, sentences, nodes, and graphs, using distributed representation have been based on the assumption that an item corresponds to one vector composed of dimensions corresponding to hidden contexts in the target. Multiple senses of an item are represented by assigning a vector to each of the domains where the item may appear or reflecting the context to the sense of the item. However, there may be multiple dis…

Exploring Weighted Property Approaches for RDF Graph Similarity Measure
Ngoc Luyen Le, Marie-H\'el\`ene Abel, Philippe Gouspillou
https://arxiv.org/abs/2404.19052

Exploring Weighted Property Approaches for RDF Graph Similarity Measure
Measuring similarity between RDF graphs is essential for various applications, including knowledge discovery, semantic web analysis, and recommender systems. However, traditional similarity measures often treat all properties equally, potentially overlooking the varying importance of different properties in different contexts. Consequently, exploring weighted property approaches for RDF graph similarity measure presents an intriguing avenue for investigation. Therefore, in this paper, we propos…

This https://arxiv.org/abs/2402.16192 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing
Aligned large language models (LLMs) are vulnerable to jailbreaking attacks, which bypass the safeguards of targeted LLMs and fool them into generating objectionable content. While initial defenses show promise against token-based threat models, there do not exist defenses that provide robustness against semantic attacks and avoid unfavorable trade-offs between robustness and nominal performance. To meet this need, we propose SEMANTICSMOOTH, a smoothing-based defense that aggregates the predict…

This https://arxiv.org/abs/2403.08748 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csRO_…

Real-time 3D semantic occupancy prediction for autonomous vehicles using memory-efficient sparse convolution
In autonomous vehicles, understanding the surrounding 3D environment of the ego vehicle in real-time is essential. A compact way to represent scenes while encoding geometric distances and semantic object information is via 3D semantic occupancy maps. State of the art 3D mapping methods leverage transformers with cross-attention mechanisms to elevate 2D vision-centric camera features into the 3D domain. However, these methods encounter significant challenges in real-time applications due to thei…

Generalizable Whole Slide Image Classification with Fine-Grained Visual-Semantic Interaction
Hao Li, Ying Chen, Yifei Chen, Wenxian Yang, Bowen Ding, Yuchen Han, Liansheng Wang, Rongshan Yu
https://arxiv.org/abs/2402.19326

Pragmatic Goal-Oriented Communications under Semantic-Effectiveness Channel Errors
Tom\'as H\"uttebr\"aucker, Mohamed Sana, Emilio Calvanese Strinati
https://arxiv.org/abs/2402.16858

Pragmatic Goal-Oriented Communications under Semantic-Effectiveness Channel Errors
In forthcoming AI-assisted 6G networks, integrating semantic, pragmatic, and goal-oriented communication strategies becomes imperative. This integration will enable sensing, transmission, and processing of exclusively pertinent task data, ensuring conveyed information possesses understandable, pragmatic semantic significance, aligning with destination needs and goals. Without doubt, no communication is error free. Within this context, besides errors stemming from typical wireless communication …

This https://arxiv.org/abs/2302.14702 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Performance Limits of a Deep Learning-Enabled Text Semantic Communication under Interference
Although deep learning (DL)-enabled semantic communication (SemCom) has emerged as a 6G enabler by minimizing irrelevant information transmission -- minimizing power usage, bandwidth consumption, and transmission delay, its benefits can be limited by radio frequency interference (RFI) that causes substantial semantic noise. Such semantic noise's impact can be alleviated using an interference-resistant and robust (IR$^2$) SemCom design, though no such design exists yet. To stimulate fundamental …

Proceedings 18th International Workshop on Logical and Semantic Frameworks, with Applications and 10th Workshop on Horn Clauses for Verification and Synthesis
Temur Kutsia (RISC, Johannes Kepler University Linz), Daniel Ventura (INF, Universidade Federal de Goi\'as), David Monniaux (CNRS - Verimag), Jos\'e F. Morales (IMDEA)
https://

Proceedings 18th International Workshop on Logical and Semantic Frameworks, with Applications and 10th Workshop on Horn Clauses for Verification and Synthesis
This volume contains * The post-proceedings of the Eighteenth Logical and Semantic Frameworks with Applications (LSFA 2023). The meeting was held on July 1-2, 2023, organised by the Sapienza Università di Roma, Italy. LSFA aims to bring researchers and students interested in theoretical and practical aspects of logical and semantic frameworks and their applications. The covered topics include proof theory, type theory and rewriting theory, specification and deduction languages, and formal se…

Exploiting Positional Bias for Query-Agnostic Generative Content in Search
Andrew Parry, Sean MacAvaney, Debasis Ganguly
https://arxiv.org/abs/2405.00469 h…

Exploiting Positional Bias for Query-Agnostic Generative Content in Search
In recent years, neural ranking models (NRMs) have been shown to substantially outperform their lexical counterparts in text retrieval. In traditional search pipelines, a combination of features leads to well-defined behaviour. However, as neural approaches become increasingly prevalent as the final scoring component of engines or as standalone systems, their robustness to malicious text and, more generally, semantic perturbation needs to be better understood. We posit that the transformer atte…

NeuralSI: Neural Design of Semantic Interaction for Interactive Deep Learning
Yali Bian, Rebecca Faust, Chris North
https://arxiv.org/abs/2402.17178 https:…

NeuralSI: Neural Design of Semantic Interaction for Interactive Deep Learning
An increasing number of studies have utilized interactive deep learning as the analytic model of visual analytics systems for complex sensemaking tasks. In these systems, traditional interactive dimensionality reduction (DR) models are commonly utilized to build a bi-directional bridge between high-dimensional deep learning representations and low-dimensional visualizations. While these systems better capture analysts' intents in the context of human-in-the-loop interactive deep learning, tradi…

This https://arxiv.org/abs/2402.16328 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIT_…

A Joint Communication and Computation Design for Probabilistic Semantic Communications
In this paper, the problem of joint transmission and computation resource allocation for a multi-user probabilistic semantic communication (PSC) network is investigated. In the considered model, users employ semantic information extraction techniques to compress their large-sized data before transmitting them to a multi-antenna base station (BS). Our model represents large-sized data through substantial knowledge graphs, utilizing shared probability graphs between the users and the BS for effic…

Learn "No" to Say "Yes" Better: Improving Vision-Language Models via Negations
Jaisidh Singh, Ishaan Shrivastava, Mayank Vatsa, Richa Singh, Aparna Bharati
https://arxiv.org/abs/2403.20312

Learn "No" to Say "Yes" Better: Improving Vision-Language Models via Negations
Existing vision-language models (VLMs) treat text descriptions as a unit, confusing individual concepts in a prompt and impairing visual semantic matching and reasoning. An important aspect of reasoning in logic and language is negations. This paper highlights the limitations of popular VLMs such as CLIP, at understanding the implications of negations, i.e., the effect of the word "not" in a given prompt. To enable evaluation of VLMs on fluent prompts with negations, we present CC-Neg, a datase…

Computational Offloading in Semantic-Aware Cloud-Edge-End Collaborative Networks
Zelin Ji, Zhijin Qin
https://arxiv.org/abs/2402.18183 https://

Computational Offloading in Semantic-Aware Cloud-Edge-End Collaborative Networks
The trend of massive connectivity pushes forward the explosive growth of end devices. The emergence of various applications has prompted a demand for pervasive connectivity and more efficient computing paradigms. On the other hand, the lack of computational capacity of the end devices restricts the implementation of the intelligent applications, and becomes a bottleneck of the multiple access for supporting massive connectivity. Mobile cloud computing (MCC) and mobile edge computing (MEC) techn…

Quantifying Semantic Query Similarity for Automated Linear SQL Grading: A Graph-based Approach
Leo K\"oberlein, Dominik Probst, Richard Lenz
https://arxiv.org/abs/2403.14441 …

Quantifying Semantic Query Similarity for Automated Linear SQL Grading: A Graph-based Approach
Quantifying the semantic similarity between database queries is a critical challenge with broad applications, ranging from query log analysis to automated educational assessment of SQL skills. Traditional methods often rely solely on syntactic comparisons or are limited to checking for semantic equivalence. This paper introduces a novel graph-based approach to measure the semantic dissimilarity between SQL queries. Queries are represented as nodes in an implicit graph, while the transitions b…

Cross-Modal Generative Semantic Communications for Mobile AIGC: Joint Semantic Encoding and Prompt Engineering
Yinqiu Liu, Hongyang Du, Dusit Niyato, Jiawen Kang, Zehui Xiong, Shiwen Mao, Ping Zhang, Xuemin Shen
https://arxiv.org/abs/2404.13898

Cross-Modal Generative Semantic Communications for Mobile AIGC: Joint Semantic Encoding and Prompt Engineering
Employing massive Mobile AI-Generated Content (AIGC) Service Providers (MASPs) with powerful models, high-quality AIGC services can become accessible for resource-constrained end users. However, this advancement, referred to as mobile AIGC, also introduces a significant challenge: users should download large AIGC outputs from the MASPs, leading to substantial bandwidth consumption and potential transmission failures. In this paper, we apply cross-modal Generative Semantic Communications (G-SemC…

PrivComp-KG : Leveraging Knowledge Graph and Large Language Models for Privacy Policy Compliance Verification
Leon Garza, Lavanya Elluri, Anantaa Kotal, Aritran Piplai, Deepti Gupta, Anupam Joshi
https://arxiv.org/abs/2404.19744 https://arxiv.org/pdf/2404.19744
arXiv:2404.19744v1 Announce Type: new
Abstract: Data protection and privacy is becoming increasingly crucial in the digital era. Numerous companies depend on third-party vendors and service providers to carry out critical functions within their operations, encompassing tasks such as data handling and storage. However, this reliance introduces potential vulnerabilities, as these vendors' security measures and practices may not always align with the standards expected by regulatory bodies. Businesses are required, often under the penalty of law, to ensure compliance with the evolving regulatory rules. Interpreting and implementing these regulations pose challenges due to their complexity. Regulatory documents are extensive, demanding significant effort for interpretation, while vendor-drafted privacy policies often lack the detail required for full legal compliance, leading to ambiguity. To ensure a concise interpretation of the regulatory requirements and compliance of organizational privacy policy with said regulations, we propose a Large Language Model (LLM) and Semantic Web based approach for privacy compliance. In this paper, we develop the novel Privacy Policy Compliance Verification Knowledge Graph, PrivComp-KG. It is designed to efficiently store and retrieve comprehensive information concerning privacy policies, regulatory frameworks, and domain-specific knowledge pertaining to the legal landscape of privacy. Using Retrieval Augmented Generation, we identify the relevant sections in a privacy policy with corresponding regulatory rules. This information about individual privacy policies is populated into the PrivComp-KG. Combining this with the domain context and rules, the PrivComp-KG can be queried to check for compliance with privacy policies by each vendor against relevant policy regulations. We demonstrate the relevance of the PrivComp-KG, by verifying compliance of privacy policy documents for various organizations.

SPINEPS -- Automatic Whole Spine Segmentation of T2-weighted MR images using a Two-Phase Approach to Multi-class Semantic and Instance Segmentation
Hendrik M\"oller, Robert Graf, Joachim Schmitt, Benjamin Keinert, Matan Atad, Anjany Sekuboyina, Felix Streckenbach, Hanna Sch\"on, Florian Kofler, Thomas Kroencke, Stefanie Bette, Stefan Willich, Thomas Keil, Thoralf Niendorf, Tobias Pischon, Beate Endemann, Bjoern Menze, Daniel Rueckert, Jan S. Kirschke

SPINEPS -- Automatic Whole Spine Segmentation of T2-weighted MR images using a Two-Phase Approach to Multi-class Semantic and Instance Segmentation
Purpose. To present SPINEPS, an open-source deep learning approach for semantic and instance segmentation of 14 spinal structures (ten vertebra substructures, intervertebral discs, spinal cord, spinal canal, and sacrum) in whole body T2w MRI. Methods. During this HIPPA-compliant, retrospective study, we utilized the public SPIDER dataset (218 subjects, 63% female) and a subset of the German National Cohort (1423 subjects, mean age 53, 49% female) for training and evaluation. We combined CT an…

PEM: Prototype-based Efficient MaskFormer for Image Segmentation
Niccol\`o Cavagnero, Gabriele Rosi, Claudia Ruttano, Francesca Pistilli, Marco Ciccone, Giuseppe Averta, Fabio Cermelli
https://arxiv.org/abs/2402.19422

PEM: Prototype-based Efficient MaskFormer for Image Segmentation
Recent transformer-based architectures have shown impressive results in the field of image segmentation. Thanks to their flexibility, they obtain outstanding performance in multiple segmentation tasks, such as semantic and panoptic, under a single unified framework. To achieve such impressive performance, these architectures employ intensive operations and require substantial computational resources, which are often not available, especially on edge devices. To fill this gap, we propose Prototy…

Sequence-level Semantic Representation Fusion for Recommender Systems
Lanling Xu, Zhen Tian, Bingqian Li, Junjie Zhang, Jinpeng Wang, Mingchen Cai, Wayne Xin Zhao
https://arxiv.org/abs/2402.18166

Sequence-level Semantic Representation Fusion for Recommender Systems
With the rapid development of recommender systems, there is increasing side information that can be employed to improve the recommendation performance. Specially, we focus on the utilization of the associated \emph{textual data} of items (eg product title) and study how text features can be effectively fused with ID features in sequential recommendation. However, there exists distinct data characteristics for the two kinds of item features, making a direct fusion method (eg adding text and ID e…

ARTiST: Automated Text Simplification for Task Guidance in Augmented Reality
Guande Wu, Jing Qian, Sonia Castelo, Shaoyu Chen, Joao Rulff, Claudio Silva
https://arxiv.org/abs/2402.18797

ARTiST: Automated Text Simplification for Task Guidance in Augmented Reality
Text presented in augmented reality provides in-situ, real-time information for users. However, this content can be challenging to apprehend quickly when engaging in cognitively demanding AR tasks, especially when it is presented on a head-mounted display. We propose ARTiST, an automatic text simplification system that uses a few-shot prompt and GPT-3 models to specifically optimize the text length and semantic content for augmented reality. Developed out of a formative study that included seve…

Semantic Information in MC: Chemotaxis Beyond Shannon
Lukas Brand, Yan Wang, Maurizio Magarini, Robert Schober, Sebastian Lotter
https://arxiv.org/abs/2402.18465

Semantic Information in MC: Chemotaxis Beyond Shannon
The recently emerging molecular communication (MC) paradigm intents to leverage communication engineering tools for the design of synthetic chemical communication systems. These systems are envisioned to operate on nanoscale and in biological environments, such as the human body, and catalyze the emergence of revolutionary applications in the context of early disease monitoring and drug targeting. However, while a plethora of theoretical (and more recently also more and more practical) MC syste…

Regular Typed Unification
Jo\~ao Barbosa, M\'ario Florido, V\'itor Santos Costa
https://arxiv.org/abs/2404.16406 https://arxi…

Regular Typed Unification
Here we define a new unification algorithm for terms interpreted in semantic domains denoted by a subclass of regular types here called deterministic regular types. This reflects our intention not to handle the semantic universe as a homogeneous collection of values, but instead, to partition it in a way that is similar to data types in programming languages. We first define the new unification algorithm which is based on constraint generation and constraint solving, and then prove its main pro…

The Era of Semantic Decoding
Maxime Peyrard, Martin Josifoski, Robert West
https://arxiv.org/abs/2403.14562 https://arxiv.org/pdf/240…

The Era of Semantic Decoding
Recent work demonstrated great promise in the idea of orchestrating collaborations between LLMs, human input, and various tools to address the inherent limitations of LLMs. We propose a novel perspective called semantic decoding, which frames these collaborative processes as optimization procedures in semantic space. Specifically, we conceptualize LLMs as semantic processors that manipulate meaningful pieces of information that we call semantic tokens (known thoughts). LLMs are among a large po…

Distributed Intelligent Integrated Sensing and Communications: The 6G-DISAC Approach
Emilio Calvanese Strinati, George C. Alexandropoulos, Madhusudan Giyyarpuram, Philippe Sehier, Sami Mekki, Vincenzo Sciancalepore, Maximilian Stark, Mohamed Sana, Benoit Denis, Maurizio Crozzoli, Navid Amani, Placido Mursia, Raffaele D Errico, Mauro Boldi, Francesca Costanzo, Francois Rivet, Henk Wymeerschx

Distributed Intelligent Integrated Sensing and Communications: The 6G-DISAC Approach
This paper introduces the concept of Distributed Intelligent integrated Sensing and Communications (DISAC), which expands the capabilities of Integrated Sensing and Communications (ISAC) towards distributed architectures. Additionally, the DISAC framework integrates novel waveform design with new semantic and goal-oriented communication paradigms, enabling ISAC technologies to transition from traditional data fusion to the semantic composition of diverse sensed and shared information. This prog…

Dynamic Relative Representations for Goal-Oriented Semantic Communications
Simone Fiorellino, Claudio Battiloro, Emilio Calvanese Strinati, Paolo Di Lorenzo
https://arxiv.org/abs/2403.16986

Dynamic Relative Representations for Goal-Oriented Semantic Communications
In future 6G wireless networks, semantic and effectiveness aspects of communications will play a fundamental role, incorporating meaning and relevance into transmissions. However, obstacles arise when devices employ diverse languages, logic, or internal representations, leading to semantic mismatches that might jeopardize understanding. In latent space communication, this challenge manifests as misalignment within high-dimensional representations where deep neural networks encode data. This pap…

NeRF-Det : Incorporating Semantic Cues and Perspective-aware Depth Supervision for Indoor Multi-View 3D Detection
Chenxi Huang, Yuenan Hou, Weicai Ye, Di Huang, Xiaoshui Huang, Binbin Lin, Deng Cai, Wanli Ouyang
https://arxiv.org/abs/2402.14464

NeRF-Det++: Incorporating Semantic Cues and Perspective-aware Depth Supervision for Indoor Multi-View 3D Detection
NeRF-Det has achieved impressive performance in indoor multi-view 3D detection by innovatively utilizing NeRF to enhance representation learning. Despite its notable performance, we uncover three decisive shortcomings in its current design, including semantic ambiguity, inappropriate sampling, and insufficient utilization of depth supervision. To combat the aforementioned problems, we present three corresponding solutions: 1) Semantic Enhancement. We project the freely available 3D segmentation…

Calculation of Femur Caput Collum Diaphyseal angle for X-Rays images using Semantic Segmentation
Deepak Bhatia, Muhammad Abdullah, Anne Querfurth, Mahdi Mantash
https://arxiv.org/abs/2404.17083

Calculation of Femur Caput Collum Diaphyseal angle for X-Rays images using Semantic Segmentation
This paper investigates the use of deep learning approaches to estimate the femur caput-collum-diaphyseal (CCD) angle from X-ray images. The CCD angle is an important measurement in the diagnosis of hip problems, and correct prediction can help in the planning of surgical procedures. Manual measurement of this angle, on the other hand, can be time-intensive and vulnerable to inter-observer variability. In this paper, we present a deep-learning algorithm that can reliably estimate the femur CCD …

Latency-Aware Generative Semantic Communications with Pre-Trained Diffusion Models
Li Qiao, Mahdi Boloursaz Mashhadi, Zhen Gao, Chuan Heng Foh, Pei Xiao, Mehdi Bennis
https://arxiv.org/abs/2403.17256

Latency-Aware Generative Semantic Communications with Pre-Trained Diffusion Models
Generative foundation AI models have recently shown great success in synthesizing natural signals with high perceptual quality using only textual prompts and conditioning signals to guide the generation process. This enables semantic communications at extremely low data rates in future wireless networks. In this paper, we develop a latency-aware semantic communications framework with pre-trained generative models. The transmitter performs multi-modal semantic decomposition on the input signal a…

This https://arxiv.org/abs/2309.11038 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csRO_…

CaveSeg: Deep Semantic Segmentation and Scene Parsing for Autonomous Underwater Cave Exploration
In this paper, we present CaveSeg - the first visual learning pipeline for semantic segmentation and scene parsing for AUV navigation inside underwater caves. We address the problem of scarce annotated training data by preparing a comprehensive dataset for semantic segmentation of underwater cave scenes. It contains pixel annotations for important navigation markers (e.g. caveline, arrows), obstacles (e.g. ground plain and overhead layers), scuba divers, and open areas for servoing. Through com…

This https://arxiv.org/abs/2403.09488 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Rectifying Demonstration Shortcut in In-Context Learning
Large language models (LLMs) are able to solve various tasks with only a few demonstrations utilizing their in-context learning (ICL) abilities. However, LLMs often rely on their pre-trained semantic priors of demonstrations rather than on the input-label relationships to proceed with ICL prediction. In this work, we term this phenomenon as the 'Demonstration Shortcut'. While previous works have primarily focused on improving ICL prediction results for predefined tasks, we aim to rectify the De…

This https://arxiv.org/abs/2402.14872 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Semantic Mirror Jailbreak: Genetic Algorithm Based Jailbreak Prompts Against Open-source LLMs
Large Language Models (LLMs), used in creative writing, code generation, and translation, generate text based on input sequences but are vulnerable to jailbreak attacks, where crafted prompts induce harmful outputs. Most jailbreak prompt methods use a combination of jailbreak templates followed by questions to ask to create jailbreak prompts. However, existing jailbreak prompt designs generally suffer from excessive semantic differences, resulting in an inability to resist defenses that use sim…

Transformer-Enhanced Motion Planner: Attention-Guided Sampling for State-Specific Decision Making
Lei Zhuang, Jingdong Zhao, Yuntao Li, Zichun Xu, Liangliang Zhao, Hong Liu
https://arxiv.org/abs/2404.19403

Transformer-Enhanced Motion Planner: Attention-Guided Sampling for State-Specific Decision Making
Sampling-based motion planning (SBMP) algorithms are renowned for their robust global search capabilities. However, the inherent randomness in their sampling mechanisms often result in inconsistent path quality and limited search efficiency. In response to these challenges, this work proposes a novel deep learning-based motion planning framework, named Transformer-Enhanced Motion Planner (TEMP), which synergizes an Environmental Information Semantic Encoder (EISE) with a Motion Planning Transfo…

SemHARQ: Semantic-Aware HARQ for Multi-task Semantic Communications
Jiangjing Hu, Fengyu Wang, Wenjun Xu, Hui Gao, Ping Zhang
https://arxiv.org/abs/2404.08490

SemHARQ: Semantic-Aware HARQ for Multi-task Semantic Communications
Intelligent task-oriented semantic communications (SemComs) have witnessed great progress with the development of deep learning (DL). In this paper, we propose a semantic-aware hybrid automatic repeat request (SemHARQ) framework for the robust and efficient transmissions of semantic features. First, to improve the robustness and effectiveness of semantic coding, a multi-task semantic encoder is proposed. Meanwhile, a feature importance ranking (FIR) method is investigated to ensure the importan…

Vision-Language Synthetic Data Enhances Echocardiography Downstream Tasks
Pooria Ashrafian, Milad Yazdani, Moein Heidari, Dena Shahriari, Ilker Hacihaliloglu
https://arxiv.org/abs/2403.19880

Vision-Language Synthetic Data Enhances Echocardiography Downstream Tasks
High-quality, large-scale data is essential for robust deep learning models in medical applications, particularly ultrasound image analysis. Diffusion models facilitate high-fidelity medical image generation, reducing the costs associated with acquiring and annotating new images. This paper utilizes recent vision-language models to produce diverse and realistic synthetic echocardiography image data, preserving key features of the original images guided by textual and semantic label maps. Specif…

Verif.ai: Towards an Open-Source Scientific Generative Question-Answering System with Referenced and Verifiable Answers
Milo\v{s} Ko\v{s}prdi\'c, Adela Ljaji\'c, Bojana Ba\v{s}aragin, Darija Medvecki, Nikola Milo\v{s}evi\'c
https://arxiv.org/abs/2402.18589

Verif.ai: Towards an Open-Source Scientific Generative Question-Answering System with Referenced and Verifiable Answers
In this paper, we present the current progress of the project Verif.ai, an open-source scientific generative question-answering system with referenced and verified answers. The components of the system are (1) an information retrieval system combining semantic and lexical search techniques over scientific papers (PubMed), (2) a fine-tuned generative model (Mistral 7B) taking top answers and generating answers with references to the papers from which the claim was derived, and (3) a verification…

Semantic Communication-assisted Physical Layer Security over Fading Wiretap Channels
Xidong Mu, Yuanwei Liu
https://arxiv.org/abs/2402.14581 https://

Semantic Communication-assisted Physical Layer Security over Fading Wiretap Channels
A novel semantic communication (SC)-assisted secrecy transmission framework is proposed. In particular, the legitimate transmitter (Tx) sends the superimposed semantic and bit stream to the legitimate receiver (Rx), where the information may be eavesdropped by the malicious node (EVE). As the EVE merely has the conventional bit-oriented communication structure, the semantic signal acts as the type of beneficial information-bearing artificial noise (AN), which not only keeps strictly confidentia…

SemHARQ: Semantic-Aware HARQ for Multi-task Semantic Communications
Jiangjing Hu, Fengyu Wang, Wenjun Xu, Hui Gao, Ping Zhang
https://arxiv.org/abs/2404.08490

SemHARQ: Semantic-Aware HARQ for Multi-task Semantic Communications
Intelligent task-oriented semantic communications (SemComs) have witnessed great progress with the development of deep learning (DL). In this paper, we propose a semantic-aware hybrid automatic repeat request (SemHARQ) framework for the robust and efficient transmissions of semantic features. First, to improve the robustness and effectiveness of semantic coding, a multi-task semantic encoder is proposed. Meanwhile, a feature importance ranking (FIR) method is investigated to ensure the importan…

GAM-Depth: Self-Supervised Indoor Depth Estimation Leveraging a Gradient-Aware Mask and Semantic Constraints
Anqi Cheng, Zhiyuan Yang, Haiyue Zhu, Kezhi Mao
https://arxiv.org/abs/2402.14354

GAM-Depth: Self-Supervised Indoor Depth Estimation Leveraging a Gradient-Aware Mask and Semantic Constraints
Self-supervised depth estimation has evolved into an image reconstruction task that minimizes a photometric loss. While recent methods have made strides in indoor depth estimation, they often produce inconsistent depth estimation in textureless areas and unsatisfactory depth discrepancies at object boundaries. To address these issues, in this work, we propose GAM-Depth, developed upon two novel components: gradient-aware mask and semantic constraints. The gradient-aware mask enables adaptive an…

Vision-Language Synthetic Data Enhances Echocardiography Downstream Tasks
Pooria Ashrafian, Milad Yazdani, Moein Heidari, Dena Shahriari, Ilker Hacihaliloglu
https://arxiv.org/abs/2403.19880

Vision-Language Synthetic Data Enhances Echocardiography Downstream Tasks
High-quality, large-scale data is essential for robust deep learning models in medical applications, particularly ultrasound image analysis. Diffusion models facilitate high-fidelity medical image generation, reducing the costs associated with acquiring and annotating new images. This paper utilizes recent vision-language models to produce diverse and realistic synthetic echocardiography image data, preserving key features of the original images guided by textual and semantic label maps. Specif…

Semantic Layering in Room Segmentation via LLMs
Taehyeon Kim, Byung-Cheol Min
https://arxiv.org/abs/2403.12920 https://arxiv.org/pdf/…

Semantic Layering in Room Segmentation via LLMs
In this paper, we introduce Semantic Layering in Room Segmentation via LLMs (SeLRoS), an advanced method for semantic room segmentation by integrating Large Language Models (LLMs) with traditional 2D map-based segmentation. Unlike previous approaches that solely focus on the geometric segmentation of indoor environments, our work enriches segmented maps with semantic data, including object identification and spatial relationships, to enhance robotic navigation. By leveraging LLMs, we provide a …

This https://arxiv.org/abs/2403.11693 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIT_…

Beamforming Design for Semantic-Bit Coexisting Communication System
Semantic communication (SemCom) is emerging as a key technology for future sixth-generation (6G) systems. Unlike traditional bit-level communication (BitCom), SemCom directly optimizes performance at the semantic level, leading to supe- rior communication efficiency. Nevertheless, the task-oriented nature of SemCom renders it challenging to completely replace BitCom. Consequently, it is desired to consider a semantic-bit coexisting communication system, where a base station (BS) serves SemCom u…

Hybridnet for depth estimation and semantic segmentation
Dalila S\'anchez-Escobedo, Xiao Lin, Josep R. Casas, Montse Pard\`as
https://arxiv.org/abs/2402.06539

Hybridnet for depth estimation and semantic segmentation
Semantic segmentation and depth estimation are two important tasks in the area of image processing. Traditionally, these two tasks are addressed in an independent manner. However, for those applications where geometric and semantic information is required, such as robotics or autonomous navigation,depth or semantic segmentation alone are not sufficient. In this paper, depth estimation and semantic segmentation are addressed together from a single input image through a hybrid convolutional netwo…

Beyond Surface Similarity: Detecting Subtle Semantic Shifts in Financial Narratives
Jiaxin Liu, Yi Yang, Kar Yan Tam
https://arxiv.org/abs/2403.14341 https…

Beyond Surface Similarity: Detecting Subtle Semantic Shifts in Financial Narratives
In this paper, we introduce the Financial-STS task, a financial domain-specific NLP task designed to measure the nuanced semantic similarity between pairs of financial narratives. These narratives originate from the financial statements of the same company but correspond to different periods, such as year-over-year comparisons. Measuring the subtle semantic differences between these paired narratives enables market stakeholders to gauge changes over time in the company's financial and operation…

Opti-Acoustic Semantic SLAM with Unknown Objects in Underwater Environments
Kurran Singh, Jungseok Hong, Nicholas R. Rypkema, John J. Leonard
https://arxiv.org/abs/2403.12837

Opti-Acoustic Semantic SLAM with Unknown Objects in Underwater Environments
Despite recent advances in semantic Simultaneous Localization and Mapping (SLAM) for terrestrial and aerial applications, underwater semantic SLAM remains an open and largely unaddressed research problem due to the unique sensing modalities and the object classes found underwater. This paper presents an object-based semantic SLAM method for underwater environments that can identify, localize, classify, and map a wide variety of marine objects without a priori knowledge of the object classes pre…

This https://arxiv.org/abs/2303.06391 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIT_…

Semantic-Aware Multi-Terminal Coding for Gaussian Mixture Sources
A novel distributed source coding model which named semantic-aware multi-terminal (MT) source coding is proposed and investigated in the paper, where multiple agents independently encode an imperceptible semantic source, while both semantic and observations are reconstructed within their respective fidelity criteria. We start from a generalized single-letter characterization of sum rate-distortion region of this problem. Furthermore, we propose a mixed MSE-Log loss framework for this model and …

Expressivity and Speech Synthesis
Andreas Triantafyllopoulos, Bj\"orn W. Schuller
https://arxiv.org/abs/2404.19363 https://arxiv.org/pdf/2404.19363
arXiv:2404.19363v1 Announce Type: new
Abstract: Imbuing machines with the ability to talk has been a longtime pursuit of artificial intelligence (AI) research. From the very beginning, the community has not only aimed to synthesise high-fidelity speech that accurately conveys the semantic meaning of an utterance, but also to colour it with inflections that cover the same range of affective expressions that humans are capable of. After many years of research, it appears that we are on the cusp of achieving this when it comes to single, isolated utterances. This unveils an abundance of potential avenues to explore when it comes to combining these single utterances with the aim of synthesising more complex, longer-term behaviours. In the present chapter, we outline the methodological advances that brought us so far and sketch out the ongoing efforts to reach that coveted next level of artificial expressivity. We also discuss the societal implications coupled with rapidly advancing expressive speech synthesis (ESS) technology and highlight ways to mitigate those risks and ensure the alignment of ESS capabilities with ethical norms.

On the Semantic Security in the General Bounded Storage Model: A New Proof
Mohammad Moltafet, Hamid R. Sadjadpour, Zouheir Rezki
https://arxiv.org/abs/2403.18935

On the Semantic Security in the General Bounded Storage Model: A New Proof
In the bounded storage model introduced by Maurer, the adversary is computationally unbounded and has a bounded storage capacity. In this model, information-theoretic secrecy is guaranteed by using a publicly available random string whose length is larger than the adversary storage capacity. The protocol proposed by Maurer is simple, from the perspective of implementation, and efficient, from the perspective of the initial secret key size and random string length. However, he provided the proof…

The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations
Aina Gar\'i Soler, Matthieu Labeau, Chlo\'e Clavel
https://arxiv.org/abs/2402.14616

The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations
When deriving contextualized word representations from language models, a decision needs to be made on how to obtain one for out-of-vocabulary (OOV) words that are segmented into subwords. What is the best way to represent these words with a single vector, and are these representations of worse quality than those of in-vocabulary words? We carry out an intrinsic evaluation of embeddings from different models on semantic similarity tasks involving OOV words. Our analysis reveals, among other int…

This https://arxiv.org/abs/2403.18933 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

SemEval-2024 Task 1: Semantic Textual Relatedness for African and Asian Languages
We present the first shared task on Semantic Textual Relatedness (STR). While earlier shared tasks primarily focused on semantic similarity, we instead investigate the broader phenomenon of semantic relatedness across 14 languages: Afrikaans, Algerian Arabic, Amharic, English, Hausa, Hindi, Indonesian, Kinyarwanda, Marathi, Moroccan Arabic, Modern Standard Arabic, Punjabi, Spanish, and Telugu. These languages originate from five distinct language families and are predominantly spoken in Africa …

Saving the legacy of Hero Ibash: Evaluating Four Language Models for Aminoacian
Yunze Xiao, Yiyang Pan
https://arxiv.org/abs/2402.18121 https://

Saving the legacy of Hero Ibash: Evaluating Four Language Models for Aminoacian
This study assesses four cutting-edge language models in the underexplored Aminoacian language. Through evaluation, it scrutinizes their adaptability, effectiveness, and limitations in text generation, semantic coherence, and contextual understanding. Uncovering insights into these models' performance in a low-resourced language, this research pioneers pathways to bridge linguistic gaps. By offering benchmarks and understanding challenges, it lays groundwork for future advancements in natural l…

This https://arxiv.org/abs/2404.08354 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Gaining More Insight into Neural Semantic Parsing with Challenging Benchmarks
The Parallel Meaning Bank (PMB) serves as a corpus for semantic processing with a focus on semantic parsing and text generation. Currently, we witness an excellent performance of neural parsers and generators on the PMB. This might suggest that such semantic processing tasks have by and large been solved. We argue that this is not the case and that performance scores from the past on the PMB are inflated by non-optimal data splits and test sets that are too easy. In response, we introduce sever…