"... the attention pattern of a single layer can be ``nearly randomized'', while preserving the functionality of the network. We also show via extensive experiments that these constructions are not merely a theoretical artifact: even after severely constraining the architecture of the model, vastly different solutions can be reached via standard training."

Attention, not time, is my most limited resource

While I'm happy to see this, I wonder if it will matter. It seems like a #fatwa to do something positive never gets as much attention - or action - as the ones to do something negative...
(Obviously, this is not unique to #Islam)

MUI: Damaging the Environment is Haram
The Ulama Council of Indonesia (MUI), has declared that all actions leading to environmental damage and contributing to the climate crisis are considered haram.

Sanders hits back at Netanyahu: ‘It is not antisemitic to hold you accountable’ | Bernie Sanders | The Guardian https://www.theguardian.com/us-news/2024/apr/27/bernie-sanders-benjamin-netanyahu-israel-gaza-war

No major American presidential candidate has talked like he now does at his rallies
— not Richard Nixon, not George Wallace, not even Donald Trump himself
Trump's demands on the world’s attention make it paradoxically easy not to pay attention to the particulars.
Most Americans, in any case, made up their minds about him one way or another long ago.
This has made Trump appear more static than he actually is
— made it harder to see how the Trump presidency,…

How Trump’s Rhetoric at Rallies Has Escalated
No major American presidential candidate has talked like he now does at his rallies — not Richard Nixon, not George Wallace, not even Donald Trump himself.

Decoding the visual attention of pathologists to reveal their level of expertise
Souradeep Chakraborty, Dana Perez, Paul Friedman, Natallia Sheuka, Constantin Friedman, Oksana Yaskiv, Rajarsi Gupta, Gregory J. Zelinsky, Joel H. Saltz, Dimitris Samaras
https://arxiv.org/abs/2403.17255

Decoding the visual attention of pathologists to reveal their level of expertise
We present a method for classifying the expertise of a pathologist based on how they allocated their attention during a cancer reading. We engage this decoding task by developing a novel method for predicting the attention of pathologists as they read whole-slide Images (WSIs) of prostate and make cancer grade classifications. Our ground truth measure of a pathologists' attention is the x, y and z (magnification) movement of their viewport as they navigated through WSIs during readings, and to …

This https://arxiv.org/abs/2310.19670 has been replaced.
link: https://scholar.google.com/scholar?q=a

Spatiotemporal Attention Enhances Lidar-Based Robot Navigation in Dynamic Environments
Foresighted robot navigation in dynamic indoor environments with cost-efficient hardware necessitates the use of a lightweight yet dependable controller. So inferring the scene dynamics from sensor readings without explicit object tracking is a pivotal aspect of foresighted navigation among pedestrians. In this paper, we introduce a spatiotemporal attention pipeline for enhanced navigation based on 2D~lidar sensor readings. This pipeline is complemented by a novel lidar-state representation tha…

“No Mr Netanyahu, it is not antisemitic or pro-Hamas to point out that, in a little over six months, your extremist government has killed over 34,000 Palestinians and wounded more than 78,000, 70% of whom are women and children,” Sanders said.
Sanders hits back at Netanyahu: ‘It is not antisemitic to hold you accountable’ | Bernie Sanders | The Guardian

Series A, Episode 10 - Breakdown
BLAKE: [Pulling Avon's attention back from Zen] Avon?
VILA: Now, wait a minute. If it's too difficult for Zen to tackle, what chance have we got?
https://blake.torpidity.net/s/110/153 B7B4

I'm with Bernie. And with all the other Jewish people who abhor the actions of the Netanyahu regime. Antisemitism is wrong, and so is antipalestinianism (who are, technically, Semitic people as well).
Sanders hits back at Netanyahu: ‘It is not antisemitic to hold you accountable’

InstructBrush: Learning Attention-based Instruction Optimization for Image Editing
Ruoyu Zhao, Qingnan Fan, Fei Kou, Shuai Qin, Hong Gu, Wei Wu, Pengcheng Xu, Mingrui Zhu, Nannan Wang, Xinbo Gao
https://arxiv.org/abs/2403.18660

InstructBrush: Learning Attention-based Instruction Optimization for Image Editing
In recent years, instruction-based image editing methods have garnered significant attention in image editing. However, despite encompassing a wide range of editing priors, these methods are helpless when handling editing tasks that are challenging to accurately describe through language. We propose InstructBrush, an inversion method for instruction-based image editing methods to bridge this gap. It extracts editing effects from exemplar image pairs as editing instructions, which are further ap…

curmudgeon they call us but it is because no one listened and get the hell off my lawn
oldschool rant but y'know it is a pain the backside when I can't describe a tech matter to an intern from within #emacs when all they know is Facebook messenger. I mean, way back when, this was not a problem, happy coexistance, to each their own. Peak Internet was fun, tho', wasn't it? Or was I just not paying attention.

JefiAtten: An Attention Based Neural Network Model for Solving Maxwell's Equations with Charge and Current Sources
Ming-Yan Sun, Peng Xu, Jun-Jie Zhang, Tai-Jiao Du, Jian-Guo Wang
https://arxiv.org/abs/2402.16920

JefiAtten: An Attention Based Neural Network Model for Solving Maxwell's Equations with Charge and Current Sources
We present JefiAtten, a novel neural network model employing the attention mechanism to solve Maxwell's equations efficiently. JefiAtten uses self-attention and cross-attention modules to understand the interplay between charge density, current density, and electromagnetic fields. Our results indicate that JefiAtten can generalize well to a range of scenarios, maintaining accuracy across various spatial distribution and handling amplitude variations. The model showcases an improvement in comput…

Attention-based Neural Network Emulators for Multi-Probe Data Vectors Part I: Forecasting the Growth-Geometry split
Kunhao Zhong, Evan Saraivanov, James Caputi, Vivian Miranda, Supranta S. Boruah, Tim Eifler, Elisabeth Krause
https://arxiv.org/abs/2402.17716

Attention-based Neural Network Emulators for Multi-Probe Data Vectors Part I: Forecasting the Growth-Geometry split
We present a new class of machine-learning emulators that accurately model the cosmic shear, galaxy-galaxy lensing, and galaxy clustering real space correlation functions in the context of Rubin Observatory year one simulated data. To illustrate its capabilities in forecasting models beyond the standard $Λ$CDM, we forecast how well LSST Year 1 data will be able to probe the consistency between geometry $Ω^{\rm geo}_\mathrm{m}$ and growth $Ω^{\rm growth}_\mathrm{m}$ dark matter densities in t…

RankMamba, Benchmarking Mamba's Document Ranking Performance in the Era of Transformers
Zhichao Xu
https://arxiv.org/abs/2403.18276 https://

RankMamba, Benchmarking Mamba's Document Ranking Performance in the Era of Transformers
Transformer structure has achieved great success in multiple applied machine learning communities, such as natural language processing (NLP), computer vision (CV) and information retrieval (IR). Transformer architecture's core mechanism -- attention requires $O(n^2)$ time complexity in training and $O(n)$ time complexity in inference. Many works have been proposed to improve the attention mechanism's scalability, such as Flash Attention and Multi-query Attention. A different line of work aims t…

The first ten episodes of Linda Teppler 's SciFi short story series, Macrodiae are free on Kindle Vella! #shortstories

Heatwaves are as deadly as other natural disasters and require policy makers and governments worldwide to take action. #IFRC and #USAid draw attention to these risks: #ActNow

Enhancing Graph Representation Learning with Attention-Driven Spiking Neural Networks
Huifeng Yin, Mingkun Xu, Jing Pei, Lei Deng
https://arxiv.org/abs/2403.17040

Enhancing Graph Representation Learning with Attention-Driven Spiking Neural Networks
Graph representation learning has become a crucial task in machine learning and data mining due to its potential for modeling complex structures such as social networks, chemical compounds, and biological systems. Spiking neural networks (SNNs) have recently emerged as a promising alternative to traditional neural networks for graph learning tasks, benefiting from their ability to efficiently encode and process temporal and spatial information. In this paper, we propose a novel approach that in…

Attention Based Molecule Generation via Hierarchical Variational Autoencoder
Divahar Sivanesan
https://arxiv.org/abs/2402.16854 https://

Attention Based Molecule Generation via Hierarchical Variational Autoencoder
Molecule generation is a task made very difficult by the complex ways in which we represent molecules computationally. A common technique used in molecular generative modeling is to use SMILES strings with recurrent neural networks built into variational autoencoders - but these suffer from a myriad of issues: vanishing gradients, long-range forgetting, and invalid molecules. In this work, we show that by combining recurrent neural networks with convolutional networks in a hierarchical manner, …

This https://arxiv.org/abs/2402.08015 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Walia-LLM: Enhancing Amharic-LLaMA by Integrating Task-Specific and Generative Datasets
Large language models (LLMs) have received a lot of attention in natural language processing (NLP) research because of their exceptional performance in understanding and generating human languages. However, low-resource languages are left behind due to the unavailability of resources. In this work, we focus on enhancing the LLaMA-2-Amharic model by integrating task-specific and generative datasets to improve language model performance for Amharic. We compile an Amharic instruction fine-tuning d…

TransFusion: Contrastive Learning with Transformers
Huanran Li, Daniel Pimentel-Alarc\'on
https://arxiv.org/abs/2403.18681 https://

TransFusion: Contrastive Learning with Transformers
This paper proposes a novel framework, TransFusion, designed to make the process of contrastive learning more analytical and explainable. TransFusion consists of attention blocks whose softmax being replaced by ReLU, and its final block's weighted-sum operation is truncated to leave the adjacency matrix as the output. The model is trained by minimizing the Jensen-Shannon Divergence between its output and the target affinity matrix, which indicates whether each pair of samples belongs to the sam…

Attention-GAN for Anomaly Detection: A Cutting-Edge Approach to Cybersecurity Threat Management
Mohammed Abo Sen
https://arxiv.org/abs/2402.15945 https://<…

Attention-GAN for Anomaly Detection: A Cutting-Edge Approach to Cybersecurity Threat Management
This paper proposes an innovative Attention-GAN framework for enhancing cybersecurity, focusing on anomaly detection. In response to the challenges posed by the constantly evolving nature of cyber threats, the proposed approach aims to generate diverse and realistic synthetic attack scenarios, thereby enriching the dataset and improving threat identification. Integrating attention mechanisms with Generative Adversarial Networks (GANs) is a key feature of the proposed method. The attention mecha…

This https://arxiv.org/abs/2310.19731 has been replaced.
link: https://scholar.google.com/scholar?q=a

ViR: Towards Efficient Vision Retention Backbones
Vision Transformers (ViTs) have attracted a lot of popularity in recent years, due to their exceptional capabilities in modeling long-range spatial dependencies and scalability for large scale training. Although the training parallelism of self-attention mechanism plays an important role in retaining great performance, its quadratic complexity baffles the application of ViTs in many scenarios which demand fast inference. This effect is even more pronounced in applications in which autoregressiv…

Sesamy, launched by Acast co-founders to let users in Sweden and Denmark purchase single articles or podcasts as a one-time transaction, plans to expand to UK (Jess Jones/City A.M.)
https://www.cityam.com/meet-the-acast-co-f

Cowboys owner Jerry Jones explains team's financial approach https://www.yardbarker.com/nfl/articles/cowboys_owner_jerry_jones_explains_teams_financial_approach/s1_14819_40296694

Cowboys owner Jerry Jones explains team's financial approach
The Cowboys’ lack of outside additions in free agency and movement in terms of extending their top players has been a key talking point this offseason. Many of Dallas’ younger in-house options will be counted on to take a step forward in 2024, while plenty of attention remains focused on the financial futures of quarterback Dak Prescott, wideout CeeDee Lamb and edge-rusher Micah Parsons. To no surprise, owner Jerry Jones has faced plenty of questions related to Dallas’ comparative inactio…

Heh, just had a light hearted talk with my Senior Director about how paying close attention to the costs of your cloud IS REALLY <expletive> IMPORTANT!
Clouds are happy, fun filled, money pits. All those super useful tools and gee-gaws exist to entrap you into their ecosystem and drain your profits.
Yes, they can provide value (worth in comparison, not cheap prices), but be mindful of the value vs cost and audit that crap every quarter. Know what, why & how long you u…

This https://arxiv.org/abs/2401.03084 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Finite sample performance of optimal treatment rule estimators with right-censored outcomes
Patient care may be improved by recommending treatments based on patient characteristics when there is treatment effect heterogeneity. Recently, there has been a great deal of attention focused on the estimation of optimal treatment rules that maximize expected outcomes. However, there has been comparatively less attention given to settings where the outcome is right-censored, especially with regard to the practical use of estimators. In this study, simulations were undertaken to assess the fin…

QKFormer: Hierarchical Spiking Transformer using Q-K Attention
Chenlin Zhou, Han Zhang, Zhaokun Zhou, Liutao Yu, Liwei Huang, Xiaopeng Fan, Li Yuan, Zhengyu Ma, Huihui Zhou, Yonghong Tian
https://arxiv.org/abs/2403.16552

QKFormer: Hierarchical Spiking Transformer using Q-K Attention
Spiking Transformers, which integrate Spiking Neural Networks (SNNs) with Transformer architectures, have attracted significant attention due to their potential for energy efficiency and high performance. However, existing models in this domain still suffer from suboptimal performance. We introduce several innovations to improve the performance: i) We propose a novel spike-form Q-K attention mechanism, tailored for SNNs, which efficiently models the importance of token or channel dimensions thr…

Former Browns teammate responds to Johnny Manziel’s allegation https://www.yardbarker.com/nfl/articles/former_browns_teammate_responds_to_johnny_manziels_allegation/s1_127_40033553

Former Browns teammate responds to Johnny Manziel’s allegation
Brian Hoyer responded on Tuesday to an allegation made by Johnny Manziel against him. Manziel made plenty of headlines last week over what he said during an appearance on Shannon Sharpe’s “Club Shay Shay” show. One thing Manziel had said that drew attention was that Hoyer was unhelpful toward him while the two were quarterbacks in Cleveland. Hoyer, Manziel and Connor Shaw were the three quarterbacks who played for the Browns in 2014, which was Manziel’s rookie season. Hoyer made 13 star…

Iwasawa invariants in residually reducible Hida families
Robert Pollack, Preston Wake
https://arXiv.org/abs/2401.14518 https://arXiv.…

Iwasawa invariants in residually reducible Hida families
We study the variation of $μ$-invariants of modular forms in a cuspidal Hida family in the case that the family intersects an Eisenstein family. We allow for intersections that occur because of "trivial zeros" (that is, because $p$ divides an Euler factor) as in Mazur's Eisenstein ideal paper, and pay special attention to the case of the 5-adic family passing through the elliptic curve $X_0(11)$.

This https://arxiv.org/abs/2212.01485 has been replaced.
link: https://scholar.google.com/scholar?q=a

A Theory of Semantic Communication
Semantic communication is an emerging research area that has gained a wide range of attention recently. Despite this growing interest, there remains a notable absence of a comprehensive and widely-accepted framework for characterizing semantic communication. This paper introduces a new conceptualization of semantic communication and formulates two fundamental problems, which we term language exploitation and language design. Our contention is that the challenge of language design can be effecti…

I'm with Bernie. And with all the other Jewish people who abhor the actions of the Netanyahu regime. Antisemitism is wrong, and so is antipalestinianism (who are, technically, Semitic people as well).
Sanders hits back at Netanyahu: ‘It is not antisemitic to hold you accountable’

Vimlike: Promising Safari Extension With Vim Keys: #Safari

This https://arxiv.org/abs/2312.03670 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Multicolor bipartite Ramsey number of double stars
For positive integers $n, m$, the double star $S(n,m)$ is the graph consisting of the disjoint union of two stars $K_{1,n}$ and $K_{1,m}$ together with an edge joining their centers. Finding monochromatic copies of double stars in edge-colored complete bipartite graphs has attracted much attention. The $k$-color bipartite Ramsey number of $ S(n,m)$, denoted by $r_{bip}(S(n,m);k)$, is the smallest integer $N$ such that, in any $k$-coloring of the edges of the complete bipartite graph $K_{N,N}$, …

Implementing and Optimizing the Scaled Dot-Product Attention on Streaming Dataflow
Gina Sohn, Nathan Zhang, Kunle Olukotun
https://arxiv.org/abs/2404.16629

Implementing and Optimizing the Scaled Dot-Product Attention on Streaming Dataflow
Transformer models serve as the backbone of many state-ofthe-art language models, and most use the scaled dot-product attention (SDPA) mechanism to capture relationships between tokens. However, the straightforward implementation of SDPA has quadratic compute and memory complexity with respect to the sequence length. On processor architectures such as GPUs and TPUs, there is a robust body of prior work. However, little work has been performed on non-processor architectures.In this work, we show…

curmudgeon they call us but it is because no one listened and get the hell off my lawn
oldschool rant but y'know it is a pain the backside when I can't describe a tech matter to an intern from within #emacs when all they know is Facebook messenger. I mean, way back when, this was not a problem, happy coexistance, to each their own. Peak Internet was fun, tho', wasn't it? Or was I just not paying attention.

🏀 @… #WCBB #NCAAW #WomensCollegeBasketball

Nonstandard Lagrangians and branched Hamiltonians: A brief review
Bijan Bagchi, Aritra Ghosh, Miloslav Znojil
https://arxiv.org/abs/2403.18801 https://

Nonstandard Lagrangians and branched Hamiltonians: A brief review
Time and again, non-conventional forms of Lagrangians have found attention in the literature. For one thing, such Lagrangians have deep connections with several aspects of nonlinear dynamics including specifically the types of the Liénard class; for another, very often the problem of their quantization opens up multiple branches of the corresponding Hamiltonians, ending up with the presence of singularities in the associated eigenfunctions. In this article, we furnish a brief review of the cla…

I kept on putting it off, but I finally took out the capacitor slowly killing most Xboxes out there.
Thanks to MattKC's video made…4 years ago for bringing it to my attention.
(desoldering is for try hards)
#Xbox #OrignalXbox

Human Simulacra: A Step toward the Personification of Large Language Models
Qiuejie Xie, Qiming Feng, Tianqi Zhang, Qingqiu Li, Yuejie Zhang, Rui Feng, Shang Gao
https://arxiv.org/abs/2402.18180

Human Simulacra: A Step toward the Personification of Large Language Models
Large language models (LLMs) are recognized as systems that closely mimic aspects of human intelligence. This capability has attracted attention from the social science community, who see the potential in leveraging LLMs to replace human participants in experiments, thereby reducing research costs and complexity. In this paper, we introduce a framework for large language models personification, including a strategy for constructing virtual characters' life stories from the ground up, a Multi-Ag…

The Kalb-Ramond field and Gravitational Parity Violation
Tucker Manton, Stephon Alexander
https://arXiv.org/abs/2401.14452 https://ar…

The Kalb-Ramond field and Gravitational Parity Violation
Gravitational wave physics can probe theories that extend beyond General Relativity. Motivated by recent attention on the Kalb-Ramond field as a dark matter candidate, in this work, we study a parity violating dimension four operator which couples the dual Riemann curvature to the 2-form field. After mapping the equations of motion for the right- and left-handed gravitational wave amplitudes to the novel parameterization recently presented in \cite{Jenks:2023pmk}, we discuss various constraints…

The Metric Distortion of Randomized Social Choice Functions: C1 Maximal Lottery Rules and Simulations
Fabian Frank, Patrick Lederer
https://arxiv.org/abs/2403.18340

The Metric Distortion of Randomized Social Choice Functions: C1 Maximal Lottery Rules and Simulations
The metric distortion of a randomized social choice function (RSCF) quantifies its worst-case approximation ratio of the optimal social cost when the voters' costs for alternatives are given by distances in a metric space. This notion has recently attracted significant attention as numerous RSCFs that aim to minimize the metric distortion have been suggested. However, such tailored voting rules usually have little appeal other than their low metric distortion. In this paper, we will thus study …

Network Flow Models for Robust Binary Optimization with Selective Adaptability
Merve Bodur, Timothy C. Y. Chan, Ian Yihang Zhu
https://arxiv.org/abs/2403.19471

Network Flow Models for Robust Binary Optimization with Selective Adaptability
Adaptive robust optimization problems have received significant attention in recent years, but remain notoriously difficult to solve when recourse decisions are discrete in nature. In this paper, we propose new reformulation techniques for adaptive robust binary optimization (ARBO) problems with objective uncertainty. Without loss of generality, we focus on ARBO problems with "selective adaptability", a term we coin to describe a common class of linking constraints between first-stage and secon…

Emergence of Large-Scale Structures in Holographic Superfluid Turbulence
Wei-Can Yang, Chuan-Yin Xia, Yu Tian, Makoto Tsubota, Hua-Bi Zeng
https://arxiv.org/abs/2402.17980

Emergence of Large-Scale Structures in Holographic Superfluid Turbulence
In two-dimensional turbulence systems, the emergence of large-scale structures holds profound physical implications, particularly as it indicates the occurrence of inverse energy cascades, thereby garnering significant attention. In this paper, we report a novel vortex clusters formation in the background of near-extreme Reissner-Nordstr$\ddot{o}$m black hole holographic model. At temperatures nearing absolute zero, we observe not only the formation of vortex clusters but also the emergence of …

EulerFormer: Sequential User Behavior Modeling with Complex Vector Attention
Zhen Tian, Wayne Xin Zhao, Changwang Zhang, Xin Zhao, Zhongrui Ma, Ji-Rong Wen
https://arxiv.org/abs/2403.17729

EulerFormer: Sequential User Behavior Modeling with Complex Vector Attention
To capture user preference, transformer models have been widely applied to model sequential user behavior data. The core of transformer architecture lies in the self-attention mechanism, which computes the pairwise attention scores in a sequence. Due to the permutation-equivariant nature, positional encoding is used to enhance the attention between token representations. In this setting, the pairwise attention scores can be derived by both semantic difference and positional difference. However,…

This https://arxiv.org/abs/2305.16616 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Channel Measurement, Modeling, and Simulation for 6G: A Survey and Tutorial
The sixth generation (6G) mobile communications have attracted substantial attention in the global research community of information and communication technologies (ICT). 6G systems are expected to support not only extended 5G usage scenarios, but also new usage scenarios, such as integrated sensing and communication (ISAC), integrated artificial intelligence (AI) and communication, and communication and ubiquitous connectivity. To realize this goal, channel characteristics must be comprehensiv…

Model Free Deep Deterministic Policy Gradient Controller for Setpoint Tracking of Non-minimum Phase Systems
Fatemeh Tavakkoli, Pouria Sarhadi, Benoit Clement, Wasif Naeem
https://arxiv.org/abs/2402.17703

Model Free Deep Deterministic Policy Gradient Controller for Setpoint Tracking of Non-minimum Phase Systems
Deep Reinforcement Learning (DRL) techniques have received significant attention in control and decision-making algorithms. Most applications involve complex decision-making systems, justified by the algorithms' computational power and cost. While model-based versions are emerging, model-free DRL approaches are intriguing for their independence from models, yet they remain relatively less explored in terms of performance, particularly in applied control. This study conducts a thorough performan…

The Supreme Court’s conservatives often accuse liberals of inventing provisions nowhere to be found in the Constitution. Now, the fingers are pointed in the other direction.
At the attention-grabbing arguments this week over Donald Trump’s claim of sweeping presidential immunity from criminal prosecution, the six-member conservative bloc seemed largely unconcerned by a key flaw in Trump’s theory:
Nothing in the Constitution explicitly mentions the concept of presidential immunit…

Heatwaves are as deadly as other natural disasters and require policy makers and governments worldwide to take action. #IFRC and #USAid draw attention to these risks: #ActNow

Russia for muslims: the new reality of Putin’s 5th term: https://benborges.xyz/2024/03/27/russia-for-muslims.html

This https://arxiv.org/abs/2311.04934 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Prompt Cache: Modular Attention Reuse for Low-Latency Inference
We present Prompt Cache, an approach for accelerating inference for large language models (LLM) by reusing attention states across different LLM prompts. Many input prompts have overlapping text segments, such as system messages, prompt templates, and documents provided for context. Our key insight is that by precomputing and storing the attention states of these frequently occurring text segments on the inference server, we can efficiently reuse them when these segments appear in user prompts.…

IIP-Mixer:Intra-Inter Patch Mixing Architecture for Battery Remaining Useful Life Prediction
Guangzai Ye, Li Feng, Jianlan Guo, Yuqiang Chen
https://arxiv.org/abs/2403.18379

IIP-Mixer:Intra-Inter Patch Mixing Architecture for Battery Remaining Useful Life Prediction
Accurately estimating the Remaining Useful Life (RUL) of lithium-ion batteries is crucial for maintaining the safe and stable operation of rechargeable battery management systems. However, this task is often challenging due to the complex temporal dynamics involved. Recently, attention-based networks, such as Transformers and Informer, have been the popular architecture in time series forecasting. Despite their effectiveness, these models with abundant parameters necessitate substantial trainin…

Few were present at the select board meeting in Littleton, N.H., last August when Carrie Gendreau, one of its members,
began to talk about a mural that had recently been painted on the side of a building downtown.
Until that moment, it had not attracted much attention. Its subject matter
— a blooming iris, dandelions, birch trees
— did not seem controversial.
But for Ms. Gendreau, 62, who was also a state senator representing northern New Hampshire,
the mu…

One Reaction to Mural Tears Small New Hampshire Town Apart
The 6,000 residents of Littleton, N.H., had found a way to coexist despite their differences — until a town official’s words set off a conflagration.

Cowboys Find Intriguing Talent in Marshawn Kneeland https://www.yardbarker.com/nfl/articles/cowboys_find_intriguing_talent_in_marshawn_kneeland/s1_17394_40290181

Cowboys Find Intriguing Talent in Marshawn Kneeland
In the 2024 NFL Draft, the Dallas Cowboys made a strategic move, selecting Marshawn Kneeland from Western Michigan as the 56th overall pick. Despite the initial surprise of bypassing higher-ranked prospects, this selection aligns with the team’s long-term vision, particularly with veteran Demarcus Lawrence nearing his 32nd birthday. Kneeland’s standout performance at Mobile caught the attention of evaluators, showcasing his natural strength and mass as a strong-side rush prospect. Transitio…

DevPhish: Exploring Social Engineering in Software Supply Chain Attacks on Developers
Hossein Siadati, Sima Jafarikhah, Elif Sahin, Terrence Brent Hernandez, Elijah Lorenzo Tripp, Denis Khryashchev
https://arxiv.org/abs/2402.18401

DevPhish: Exploring Social Engineering in Software Supply Chain Attacks on Developers
The Software Supply Chain (SSC) has captured considerable attention from attackers seeking to infiltrate systems and undermine organizations. There is evidence indicating that adversaries utilize Social Engineering (SocE) techniques specifically aimed at software developers. That is, they interact with developers at critical steps in the Software Development Life Cycle (SDLC), such as accessing Github repositories, incorporating code dependencies, and obtaining approval for Pull Requests (PR) t…

A Novel Approach to Chest X-ray Lung Segmentation Using U-net and Modified Convolutional Block Attention Module
Mohammad Ali Labbaf Khaniki, Mohammad Manthouri
https://arxiv.org/abs/2404.14322

A Novel Approach to Chest X-ray Lung Segmentation Using U-net and Modified Convolutional Block Attention Module
Lung segmentation in chest X-ray images is of paramount importance as it plays a crucial role in the diagnosis and treatment of various lung diseases. This paper presents a novel approach for lung segmentation in chest X-ray images by integrating U-net with attention mechanisms. The proposed method enhances the U-net architecture by incorporating a Convolutional Block Attention Module (CBAM), which unifies three distinct attention mechanisms: channel attention, spatial attention, and pixel atte…

This https://arxiv.org/abs/2403.15360 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCV_…

SiMBA: Simplified Mamba-Based Architecture for Vision and Multivariate Time series
Transformers have widely adopted attention networks for sequence mixing and MLPs for channel mixing, playing a pivotal role in achieving breakthroughs across domains. However, recent literature highlights issues with attention networks, including low inductive bias and quadratic complexity concerning input sequence length. State Space Models (SSMs) like S4 and others (Hippo, Global Convolutions, liquid S4, LRU, Mega, and Mamba), have emerged to address the above issues to help handle longer seq…

I kept on putting it off, but I finally took out the capacitor slowly killing most Xboxes out there.
Thanks to MattKC's video made…4 years ago for bringing it to my attention.
(desoldering is for try hards)
#Xbox #OrignalXbox

This https://arxiv.org/abs/2312.17296 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Structured Packing in LLM Training Improves Long Context Utilization
Recent developments in long-context large language models have attracted considerable attention. Yet, their real-world applications are often hindered by ineffective context information use. This work shows that structuring training data to increase semantic interdependence is an effective strategy for optimizing context utilization. To this end, we introduce Structured Packing for Long Context (SPLiCe), a method for creating training examples by using information retrieval methods to collate m…

Codebook-enabled Generative End-to-end Semantic Communication Powered by Transformer
Peigen Ye, Yaping Sun, Shumin Yao, Hao Chen, Xiaodong Xu, Shuguang Cui
https://arxiv.org/abs/2402.16868

Codebook-enabled Generative End-to-end Semantic Communication Powered by Transformer
Codebook-based generative semantic communication attracts increasing attention, since only indices are required to be transmitted when the codebook is shared between transmitter and receiver. However, due to the fact that the semantic relations among code vectors are not necessarily related to the distance of the corresponding code indices, the performance of the codebook-enabled semantic communication system is susceptible to the channel noise. Thus, how to improve the system robustness agains…

Machine Unlearning in Large Language Models
Kongyang Chen, Zixin Wang, Bing Mi, Waixi Liu, Shaowei Wang, Xiaojun Ren, Jiaxing Shen
https://arxiv.org/abs/2404.16841

Machine Unlearning in Large Language Models
Recently, large language models (LLMs) have emerged as a notable field, attracting significant attention for its ability to automatically generate intelligent contents for various application domains. However, LLMs still suffer from significant security and privacy issues. For example, LLMs might expose user privacy from hacking attacks or targeted prompts. To address this problem, this paper introduces a novel machine unlearning framework into LLMs. Our objectives are to make LLMs not produce …

This https://arxiv.org/abs/2309.16120 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSE_…

Test-Case-Driven Programming Understanding in Large Language Models for Better Code Generation
Code generation is to automatically generate source code conforming to a given programming specification, which has received extensive attention especially with the development of large language models (LLMs). Due to the inherent difficulty of code generation, the code generated by LLMs may be not aligned with the specification. To improve the performance of LLMs in code generation, some thought-eliciting prompting techniques have been proposed to guide LLMs for specification understanding. How…

This https://arxiv.org/abs/2403.11505 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Covid-19 detection from CT scans using EfficientNet and Attention mechanism
Manual diagnosis and analysis of COVID-19 through the examination of lung Computed Tomography (CT) scan images by physicians tends to result in inefficiency, especially with high patient volumes and numerous images per patient. We address the need for automation by developing a deep learning model-based pipeline for COVID-19 detection from CT scan images of the lungs. The Domain adaptation, Explainability, and Fairness in AI for Medical Image Analysis Workshop and COVID-19 Diagnosis Competition…

T\"urk\c{c}e Dil Modellerinin Performans Kar\c{s}{\i}la\c{s}t{\i}rmas{\i} Performance Comparison of Turkish Language Models
Eren Dogan, M. Egemen Uzun, Atahan Uz, H. Emre Seyrek, Ahmed Zeer, Ezgi Sevi, H. Toprak Kesgin, M. Kaan Yuce, M. Fatih Amasyali
https://arxiv.org/abs/2404.17010

Türkçe Dil Modellerinin Performans Karşılaştırması Performance Comparison of Turkish Language Models
The developments that language models have provided in fulfilling almost all kinds of tasks have attracted the attention of not only researchers but also the society and have enabled them to become products. There are commercially successful language models available. However, users may prefer open-source language models due to cost, data privacy, or regulations. Yet, despite the increasing number of these models, there is no comprehensive comparison of their performance for Turkish. This study…

Random Aggregate Beamforming for Over-the-Air Federated Learning in Large-Scale Networks
Chunmei Xu, Shengheng Liu, Yongming Huang, Bjorn Ottersten, Dusit Niyato
https://arxiv.org/abs/2403.18946

Random Aggregate Beamforming for Over-the-Air Federated Learning in Large-Scale Networks
At present, there is a trend to deploy ubiquitous artificial intelligence (AI) applications at the edge of the network. As a promising framework that enables secure edge intelligence, federated learning (FL) has received widespread attention, and over-the-air computing (AirComp) has been integrated to further improve the communication efficiency. In this paper, we consider a joint device selection and aggregate beamforming design with the objectives of minimizing the aggregate error and maximiz…

This https://arxiv.org/abs/2402.14096 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSE_…

EyeTrans: Merging Human and Machine Attention for Neural Code Summarization
Neural code summarization leverages deep learning models to automatically generate brief natural language summaries of code snippets. The development of Transformer models has led to extensive use of attention during model design. While existing work has primarily and almost exclusively focused on static properties of source code and related structural representations like the Abstract Syntax Tree (AST), few studies have considered human attention, that is, where programmers focus while examini…

This https://arxiv.org/abs/2404.15848 has been replaced.
link: https://scholar.google.com/scholar?q=a

Detecting Conceptual Abstraction in LLMs
We present a novel approach to detecting noun abstraction within a large language model (LLM). Starting from a psychologically motivated set of noun pairs in taxonomic relationships, we instantiate surface patterns indicating hypernymy and analyze the attention matrices produced by BERT. We compare the results to two sets of counterfactuals and show that we can detect hypernymy in the abstraction mechanism, which cannot solely be related to the distributional similarity of noun pairs. Our findi…

This https://arxiv.org/abs/2403.11505 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Covid-19 detection from CT scans using EfficientNet and Attention mechanism
Manual diagnosis and analysis of COVID-19 through the examination of lung Computed Tomography (CT) scan images by physicians tends to result in inefficiency, especially with high patient volumes and numerous images per patient. We address the need for automation by developing a deep learning model-based pipeline for COVID-19 detection from CT scan images of the lungs. The Domain adaptation, Explainability, and Fairness in AI for Medical Image Analysis Workshop and COVID-19 Diagnosis Competition…

Exploring the Market Dynamics of Liquid Staking Derivatives (LSDs)
Xihan Xiong, Zhipeng Wang, Qin Wang
https://arxiv.org/abs/2402.17748 https://

Exploring the Market Dynamics of Liquid Staking Derivatives (LSDs)
Staking has emerged as a crucial concept following Ethereum's transition to Proof-of-Stake consensus. The introduction of Liquid Staking Derivatives (LSDs) has effectively addressed the illiquidity issue associated with solo staking, gaining significant market attention. This paper analyzes the LSD market dynamics from the perspectives of both liquidity takers (LTs) and liquidity providers (LPs). We first quantify the price discrepancy between the LSD primary and secondary markets. Then we inve…

Are Large Language Models Good at Utility Judgments?
Hengran Zhang, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yixing Fan, Xueqi Cheng
https://arxiv.org/abs/2403.19216

Are Large Language Models Good at Utility Judgments?
Retrieval-augmented generation (RAG) is considered to be a promising approach to alleviate the hallucination issue of large language models (LLMs), and it has received widespread attention from researchers recently. Due to the limitation in the semantic understanding of retrieval models, the success of RAG heavily lies on the ability of LLMs to identify passages with utility. Recent efforts have explored the ability of LLMs to assess the relevance of passages in retrieval, but there has been li…

Few-Shot Cross-System Anomaly Trace Classification for Microservice-based systems
Yuqing Wang, Mika V. Mantyl\"a, Serge Demeyer, Mutlu Beyazit, Joanna Kisaakye, Jesse Nyyss\"ol\"a
https://arxiv.org/abs/2403.18998

Few-Shot Cross-System Anomaly Trace Classification for Microservice-based systems
Microservice-based systems (MSS) may experience failures in various fault categories due to their complex and dynamic nature. To effectively handle failures, AIOps tools utilize trace-based anomaly detection and root cause analysis. In this paper, we propose a novel framework for few-shot abnormal trace classification for MSS. Our framework comprises two main components: (1) Multi-Head Attention Autoencoder for constructing system-specific trace representations, which enables (2) Transformer En…

This https://arxiv.org/abs/2402.16696 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Look Before You Leap: Towards Decision-Aware and Generalizable Tool-Usage for Large Language Models
Tool-augmented large language models (LLMs) are attracting widespread attention when accessing up-to-date knowledge and alleviating hallucination issues. Nowadays, advanced closed-source LLMs (e.g., ChatGPT) have demonstrated surprising tool-usage capabilities through prompting and in-context learning techniques. To empower the capabilities of open-source LLMs (e.g., LLaMA) in manipulating tools, current efforts focus on either template-driven or token-triggered tool-usage. However, the former …

This https://arxiv.org/abs/2308.09831 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Cross-modality Attention-based Multimodal Fusion for Non-small Cell Lung Cancer (NSCLC) Patient Survival Prediction
Cancer prognosis and survival outcome predictions are crucial for therapeutic response estimation and for stratifying patients into various treatment groups. Medical domains concerned with cancer prognosis are abundant with multiple modalities, including pathological image data and non-image data such as genomic information. To date, multimodal learning has shown potential to enhance clinical prediction model performance by extracting and aggregating information from different modalities of the…

Analyzing the concept of technical debt in the context of agile software development: A systematic literature review
Woubshet Nema Behutiye, Pilar Rodriguez, Markku Oivo, Ayse Tosun
https://arXiv.org/abs/2401.14882

Analyzing the concept of technical debt in the context of agile software development: A systematic literature review
Technical debt (TD) is a metaphor that is used to communicate the consequences of poor software development practices to non-technical stakeholders. In recent years, it has gained significant attention in agile software development (ASD). The purpose of this study is to analyze and synthesize the state of the art of TD, and its causes, consequences, and management strategies in the context of ASD. Using a systematic literature review (SLR), 38 primary studies, out of 346 studies, were identifie…

This https://arxiv.org/abs/2402.16458 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

D-XCB: Data-independent Debiasing for Fair and Accurate Transformer-based Cyberbullying Detection
Swear words are a common proxy to collect datasets with cyberbullying incidents. Our focus is on measuring and mitigating biases derived from spurious associations between swear words and incidents occurring as a result of such data collection strategies. After demonstrating and quantifying these biases, we introduce ID-XCB, the first data-independent debiasing technique that combines adversarial training, bias constraints and debias fine-tuning approach aimed at alleviating model attention to …

This https://arxiv.org/abs/2306.14287 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Efficient Contextformer: Spatio-Channel Window Attention for Fast Context Modeling in Learned Image Compression
Entropy estimation is essential for the performance of learned image compression. It has been demonstrated that a transformer-based entropy model is of critical importance for achieving a high compression ratio, however, at the expense of a significant computational effort. In this work, we introduce the Efficient Contextformer (eContextformer) - a computationally efficient transformer-based autoregressive context model for learned image compression. The eContextformer efficiently fuses the pat…

Analyzing the concept of technical debt in the context of agile software development: A systematic literature review
Woubshet Nema Behutiye, Pilar Rodriguez, Markku Oivo, Ayse Tosun
https://arXiv.org/abs/2401.14882

Analyzing the concept of technical debt in the context of agile software development: A systematic literature review
Technical debt (TD) is a metaphor that is used to communicate the consequences of poor software development practices to non-technical stakeholders. In recent years, it has gained significant attention in agile software development (ASD). The purpose of this study is to analyze and synthesize the state of the art of TD, and its causes, consequences, and management strategies in the context of ASD. Using a systematic literature review (SLR), 38 primary studies, out of 346 studies, were identifie…

Onboard deep lossless and near-lossless predictive coding of hyperspectral images with line-based attention
Diego Valsesia, Tiziano Bianchi, Enrico Magli
https://arxiv.org/abs/2403.17677

Onboard deep lossless and near-lossless predictive coding of hyperspectral images with line-based attention
Deep learning methods have traditionally been difficult to apply to compression of hyperspectral images onboard of spacecrafts, due to the large computational complexity needed to achieve adequate representational power, as well as the lack of suitable datasets for training and testing. In this paper, we depart from the traditional autoencoder approach and we design a predictive neural network, called LineRWKV, that works recursively line-by-line to limit memory consumption. In order to achieve…

Ansible Lightspeed: A Code Generation Service for IT Automation
Priyam Sahoo, Saurabh Pujar, Ganesh Nalawade, Richard Gebhardt, Louis Mandel, Luca Buratti
https://arxiv.org/abs/2402.17442

Ansible Lightspeed: A Code Generation Service for IT Automation
The availability of Large Language Models (LLMs) which can generate code, has made it possible to create tools that improve developer productivity. Integrated development environments or IDEs which developers use to write software are often used as an interface to interact with LLMs. Although many such tools have been released, almost all of them focus on general-purpose programming languages. Domain-specific languages, such as those crucial for IT automation, have not received much attention. …

Joint chest X-ray diagnosis and clinical visual attention prediction with multi-stage cooperative learning: enhancing interpretability
Zirui Qiu, Hassan Rivaz, Yiming Xiao
https://arxiv.org/abs/2403.16970

Joint chest X-ray diagnosis and clinical visual attention prediction with multi-stage cooperative learning: enhancing interpretability
As deep learning has become the state-of-the-art for computer-assisted diagnosis, interpretability of the automatic decisions is crucial for clinical deployment. While various methods were proposed in this domain, visual attention maps of clinicians during radiological screening offer a unique asset to provide important insights and can potentially enhance the quality of computer-assisted diagnosis. With this paper, we introduce a novel deep-learning framework for joint disease diagnosis and pr…

This https://arxiv.org/abs/2306.00003 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Detecting Heart Disease from Multi-View Ultrasound Images via Supervised Attention Multiple Instance Learning
Aortic stenosis (AS) is a degenerative valve condition that causes substantial morbidity and mortality. This condition is under-diagnosed and under-treated. In clinical practice, AS is diagnosed with expert review of transthoracic echocardiography, which produces dozens of ultrasound images of the heart. Only some of these views show the aortic valve. To automate screening for AS, deep networks must learn to mimic a human expert's ability to identify views of the aortic valve then aggregate acr…

This https://arxiv.org/abs/2310.08080 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

RT-SRTS: Angle-Agnostic Real-Time Simultaneous 3D Reconstruction and Tumor Segmentation from Single X-Ray Projection
Radiotherapy is one of the primary treatment methods for tumors, but the organ movement caused by respiration limits its accuracy. Recently, 3D imaging from a single X-ray projection has received extensive attention as a promising approach to address this issue. However, current methods can only reconstruct 3D images without directly locating the tumor and are only validated for fixed-angle imaging, which fails to fully meet the requirements of motion control in radiotherapy. In this study, a n…

This https://arxiv.org/abs/2310.08080 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

RT-SRTS: Angle-Agnostic Real-Time Simultaneous 3D Reconstruction and Tumor Segmentation from Single X-Ray Projection
Radiotherapy is one of the primary treatment methods for tumors, but the organ movement caused by respiration limits its accuracy. Recently, 3D imaging from a single X-ray projection has received extensive attention as a promising approach to address this issue. However, current methods can only reconstruct 3D images without directly locating the tumor and are only validated for fixed-angle imaging, which fails to fully meet the requirements of motion control in radiotherapy. In this study, a n…