Tootfinder

Flexible Operator Fusion for Fast Sparse Transformer with Diverse Masking on GPU
Wenhao Dai, Haodong Deng, Mengfei Rong, Xinyu Yang, Hongyu Liu, Fangxin Liu, Hailong Yang, Weifeng Liu, Qingxiao Sun
https://arxiv.org/abs/2506.06095

Flexible Operator Fusion for Fast Sparse Transformer with Diverse Masking on GPU
Large language models are popular around the world due to their powerful understanding capabilities. As the core component of LLMs, accelerating Transformer through parallelization has gradually become a hot research topic. Mask layers introduce sparsity into Transformer to reduce calculations. However, previous works rarely focus on the performance optimization of sparse Transformer. Moreover, rule-based mechanisms ignore the fusion opportunities of mixed-type operators and fail to adapt to va…

Applying Informer for Option Pricing: A Transformer-Based Approach
Feliks Ba\'nka, Jaros{\l}aw A. Chudziak
https://arxiv.org/abs/2506.05565 https://

Applying Informer for Option Pricing: A Transformer-Based Approach
Accurate option pricing is essential for effective trading and risk management in financial markets, yet it remains challenging due to market volatility and the limitations of traditional models like Black-Scholes. In this paper, we investigate the application of the Informer neural network for option pricing, leveraging its ability to capture long-term dependencies and dynamically adjust to market fluctuations. This research contributes to the field of financial forecasting by introducing Info…

Unisoma: A Unified Transformer-based Solver for Multi-Solid Systems
Shilong Tao, Zhe Feng, Haonan Sun, Zhanxing Zhu, Yunhuai Liu
https://arxiv.org/abs/2506.06021

Unisoma: A Unified Transformer-based Solver for Multi-Solid Systems
Multi-solid systems are foundational to a wide range of real-world applications, yet modeling their complex interactions remains challenging. Existing deep learning methods predominantly rely on implicit modeling, where the factors influencing solid deformation are not explicitly represented but are instead indirectly learned. However, as the number of solids increases, these methods struggle to accurately capture intricate physical interactions. In this paper, we introduce a novel explicit mod…

teaching a transformer net to approximate ray-tracing
input is a sequence of triangle data (limited to 4k triangles), and a sequence of 8x8 radiance pixels representing the camera (limited to 512x512 resolution)
output is a sequence of 8x8 pixels of the rendered scene. training was ~10 days. generation is ~100ms (NVIDIA A100)
the results are pretty good. scaling will be a challenge (since transformers are quadratic in sequence length)

STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesis
Jiatao Gu, Tianrong Chen, David Berthelot, Huangjie Zheng, Yuyang Wang, Ruixiang Zhang, Laurent Dinh, Miguel Angel Bautista, Josh Susskind, Shuangfei Zhai
https://arxiv.org/abs/2506.06276

STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesis
We present STARFlow, a scalable generative model based on normalizing flows that achieves strong performance in high-resolution image synthesis. The core of STARFlow is Transformer Autoregressive Flow (TARFlow), which combines the expressive power of normalizing flows with the structured modeling capabilities of Autoregressive Transformers. We first establish the theoretical universality of TARFlow for modeling continuous distributions. Building on this foundation, we introduce several key arch…

Text-to-LoRA: Instant Transformer Adaption
Rujikorn Charakorn, Edoardo Cetin, Yujin Tang, Robert Tjarko Lange
https://arxiv.org/abs/2506.06105 https://

Text-to-LoRA: Instant Transformer Adaption
While Foundation Models provide a general tool for rapid content creation, they regularly require task-specific adaptation. Traditionally, this exercise involves careful curation of datasets and repeated fine-tuning of the underlying model. Fine-tuning techniques enable practitioners to adapt foundation models for many new applications but require expensive and lengthy training while being notably sensitive to hyper-parameter choices. To overcome these limitations, we introduce Text-to-LoRA (T2…

Is BERTopic Better than PLSA for Extracting Key Topics in Aviation Safety Reports?
Aziida Nanyonga, Joiner Keith, Turhan Ugur, Wild Graham
https://arxiv.org/abs/2506.06328

Is BERTopic Better than PLSA for Extracting Key Topics in Aviation Safety Reports?
This study compares the effectiveness of BERTopic and Probabilistic Latent Semantic Analysis (PLSA) in extracting meaningful topics from aviation safety reports aiming to enhance the understanding of patterns in aviation incident data. Using a dataset of over 36,000 National Transportation Safety Board (NTSB) reports from 2000 to 2020, BERTopic employed transformer based embeddings and hierarchical clustering, while PLSA utilized probabilistic modelling through the Expectation-Maximization (EM)…

This https://arxiv.org/abs/2506.03439 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Optimizing Software Defined Battery Systems for Transformer Protection
Residential electric vehicle charging causes large spikes in electricity demand that risk violating neighborhood transformer power limits. Battery energy storage systems reduce these transformer limit violations, but operating them individually is not cost-optimal. Instead of individual optimization, aggregating, or sharing, these batteries leads to cost-optimal performance, but homeowners must relinquish battery control. This paper leverages virtualization to propose battery sharing optimizati…

Mitigating Catastrophic Forgetting with Adaptive Transformer Block Expansion in Federated Fine-Tuning
Yujia Huo, Jianchun Liu, Hongli Xu, Zhenguo Ma, Shilong Wang, Liusheng Huang
https://arxiv.org/abs/2506.05977

Mitigating Catastrophic Forgetting with Adaptive Transformer Block Expansion in Federated Fine-Tuning
Federated fine-tuning (FedFT) of large language models (LLMs) has emerged as a promising solution for adapting models to distributed data environments while ensuring data privacy. Existing FedFT methods predominantly utilize parameter-efficient fine-tuning (PEFT) techniques to reduce communication and computation overhead. However, they often fail to adequately address the catastrophic forgetting, a critical challenge arising from continual adaptation in distributed environments. The tradit…

Enhancing Automatic PT Tagging for MEDLINE Citations Using Transformer-Based Models
Victor H. Cid, James Mork
https://arxiv.org/abs/2506.03321 https://

Enhancing Automatic PT Tagging for MEDLINE Citations Using Transformer-Based Models
We investigated the feasibility of predicting Medical Subject Headings (MeSH) Publication Types (PTs) from MEDLINE citation metadata using pre-trained Transformer-based models BERT and DistilBERT. This study addresses limitations in the current automated indexing process, which relies on legacy NLP algorithms. We evaluated monolithic multi-label classifiers and binary classifier ensembles to enhance the retrieval of biomedical literature. Results demonstrate the potential of Transformer models …

SatelliteFormula: Multi-Modal Symbolic Regression from Remote Sensing Imagery for Physics Discovery
Zhenyu Yu, Mohd. Yamani Idna Idris, Pei Wang, Yuelong Xia, Fei Ma, Rizwan Qureshi
https://arxiv.org/abs/2506.06176

SatelliteFormula: Multi-Modal Symbolic Regression from Remote Sensing Imagery for Physics Discovery
We propose SatelliteFormula, a novel symbolic regression framework that derives physically interpretable expressions directly from multi-spectral remote sensing imagery. Unlike traditional empirical indices or black-box learning models, SatelliteFormula combines a Vision Transformer-based encoder for spatial-spectral feature extraction with physics-guided constraints to ensure consistency and interpretability. Existing symbolic regression methods struggle with the high-dimensional complexity of…

Enhancing Text Comprehension for Dyslexic Readers: A 3D Semantic Visualization Approach Using Transformer Mode
Zhengyang Li
https://arxiv.org/abs/2506.03731

Enhancing Text Comprehension for Dyslexic Readers: A 3D Semantic Visualization Approach Using Transformer Mode
Dyslexic individuals often face significant challenges with traditional reading, particularly when engaging with complex texts such as mystery novels. These texts typically demand advanced narrative tracking and information integration skills, making it difficult for dyslexic readers to fully comprehend the content. However, research indicates that while dyslexic individuals may struggle with textual processing, they often possess strong spatial imagination abilities. Leveraging this strength, …

Efficient Tactile Perception with Soft Electrical Impedance Tomography and Pre-trained Transformer
Huazhi Dong (Sharel), Ronald B. Liu (Sharel), Sihao Teng (Sharel), Delin Hu (Sharel), Peisan (Sharel), E, Francesco Giorgio-Serchi, Yunjie Yang
https://arxiv.org/abs/2506.02824

Efficient Tactile Perception with Soft Electrical Impedance Tomography and Pre-trained Transformer
Tactile sensing is fundamental to robotic systems, enabling interactions through physical contact in multiple tasks. Despite its importance, achieving high-resolution, large-area tactile sensing remains challenging. Electrical Impedance Tomography (EIT) has emerged as a promising approach for large-area, distributed tactile sensing with minimal electrode requirements which can lend itself to addressing complex contact problems in robotics. However, existing EIT-based tactile reconstruction meth…

External Attention Transformer: A Robust AI Model for Identifying Initial Eccentricity Signatures in Binary Black Hole Events in Simulated Advanced LIGO Data
Elahe Khalouei, Cristiano G. Sabiu, Hyung Mok Lee, A. Gopakumar
https://arxiv.org/abs/2506.03634

External Attention Transformer: A Robust AI Model for Identifying Initial Eccentricity Signatures in Binary Black Hole Events in Simulated Advanced LIGO Data
Initial orbital eccentricities of gravitational wave (GW) events associated with merging binary black holes (BBHs) should provide clues to their formation scenarios, mainly because various BBH formation channels predict distinct eccentricity distributions. However, searching for inspiral GWs from eccentric BBHs is computationally challenging due to sophisticated approaches to model such GW events. This ensures that Bayesian parameter estimation methods to characterize such events are computatio…

This https://arxiv.org/abs/2506.02576 has been replaced.
link: https://scholar.google.com/scholar?q=a

ADFormer: Aggregation Differential Transformer for Passenger Demand Forecasting
Passenger demand forecasting helps optimize vehicle scheduling, thereby improving urban efficiency. Recently, attention-based methods have been used to adequately capture the dynamic nature of spatio-temporal data. However, existing methods that rely on heuristic masking strategies cannot fully adapt to the complex spatio-temporal correlations, hindering the model from focusing on the right context. These works also overlook the high-level correlations that exist in the real world. Effectively …

Optimizing Software Defined Battery Systems for Transformer Protection
Sonia Martin, Obidike Nnorom Jr., Philip Levis, Ram Rajagopal
https://arxiv.org/abs/2506.03439

Optimizing Software Defined Battery Systems for Transformer Protection
Residential electric vehicle charging causes large spikes in electricity demand that risk violating neighborhood transformer power limits. Battery energy storage systems reduce these transformer limit violations, but operating them individually is not cost-optimal. Instead of individual optimization, aggregating, or sharing, these batteries leads to cost-optimal performance, but homeowners must relinquish battery control. This paper leverages virtualization to propose battery sharing optimizati…

A Transformer-Based Neural Network for Optimal Deterministic-Allocation and Anonymous Joint Auction Design
Zhen Zhang, Luowen Liu, Wanzhi Zhang, Zitian Guo, Kun Huang, Qi Qi, Qiang Liu, Xingxing Wang
https://arxiv.org/abs/2506.02435

A Transformer-Based Neural Network for Optimal Deterministic-Allocation and Anonymous Joint Auction Design
With the advancement of machine learning, an increasing number of studies are employing automated mechanism design (AMD) methods for optimal auction design. However, all previous AMD architectures designed to generate optimal mechanisms that satisfy near dominant strategy incentive compatibility (DSIC) fail to achieve deterministic allocation, and some also lack anonymity, thereby impacting the efficiency and fairness of advertising allocation. This has resulted in a notable discrepancy between…

Life Sequence Transformer: Generative Modelling for Counterfactual Simulation
Alberto Cabezas, Carlotta Montorsi
https://arxiv.org/abs/2506.01874 https://

Life Sequence Transformer: Generative Modelling for Counterfactual Simulation
Social sciences rely on counterfactual analysis using surveys and administrative data, generally depending on strong assumptions or the existence of suitable control groups, to evaluate policy interventions and estimate causal effects. We propose a novel approach that leverages the Transformer architecture to simulate counterfactual life trajectories from large-scale administrative records. Our contributions are: the design of a novel encoding method that transforms longitudinal administrative …

This https://arxiv.org/abs/2502.03503 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Analyzing limits for in-context learning
We examine limits of in-context learning (ICL) in transformer models trained from scratch, focusing on function approximation tasks as a controlled setting to uncover fundamental behaviors. While we show empirically that transformer models can generalize, approximating unseen classes of polynomial (non linear) functions, they cannot generalize beyond certain values. We provide both empirical and mathematical arguments explaining that these limitations stem from architectural components, namely …

TRiMM: Transformer-Based Rich Motion Matching for Real-Time multi-modal Interaction in Digital Humans
Yueqian Guo, Tianzhao Li, Xin Lyu, Jiehaolin Chen, Zhaohan Wang, Sirui Xiao, Yurun Chen, Yezi He, Helin Li, Fan Zhang
https://arxiv.org/abs/2506.01077

TRiMM: Transformer-Based Rich Motion Matching for Real-Time multi-modal Interaction in Digital Humans
Large Language Model (LLM)-driven digital humans have sparked a series of recent studies on co-speech gesture generation systems. However, existing approaches struggle with real-time synthesis and long-text comprehension. This paper introduces Transformer-Based Rich Motion Matching (TRiMM), a novel multi-modal framework for real-time 3D gesture generation. Our method incorporates three modules: 1) a cross-modal attention mechanism to achieve precise temporal alignment between speech and gesture…

This https://arxiv.org/abs/2503.02407 has been replaced.
initial toot: https://mastoxiv.page/@a…

Wyckoff Transformer: Generation of Symmetric Crystals
Crystal symmetry plays a fundamental role in determining its physical, chemical, and electronic properties such as electrical and thermal conductivity, optical and polarization behavior, and mechanical strength. Almost all known crystalline materials have internal symmetry. However, this is often inadequately addressed by existing generative models, making the consistent generation of stable and symmetrically valid crystal structures a significant challenge. We introduce WyFormer, a generative …

This https://arxiv.org/abs/2505.17452 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_…

Deblending Overlapping Galaxies in DECaLS Using Transformer-Based Algorithm: A Method Combining Multiple Bands and Data Types
In large-scale galaxy surveys, particularly deep ground-based photometric studies, galaxy blending is inevitable and poses a potential primary systematic uncertainty for upcoming surveys. Current deblenders predominantly rely on analytical modeling of galaxy profiles, facing limitations due to inflexible and imprecise models. We present a novel approach using a U-net structured transformer-based network for deblending astronomical images, which we term the CAT-deblender. It was trained using bo…

This https://arxiv.org/abs/2504.14483 has been replaced.
initial toot: https://mastoxiv.page/@arX…

$Σ$-Attention: A Transformer-based operator learning framework for self-energy in strongly correlated systems
We introduce $Σ$-Attention, a Transformer-based operator-learning framework to address a key computational challenge in correlated materials. Our approach utilizes an Encoder-Only Transformer as an ansatz to approximate the self-energy operator of strongly correlated electronic systems. By creating a batched dataset that combines results from three complementary approaches: many-body perturbation theory, strong-coupling expansion, and exact diagonalization, each effective in specific parameter…

Hybrid SLC-MLC RRAM Mixed-Signal Processing-in-Memory Architecture for Transformer Acceleration via Gradient Redistribution
Chang Eun Song, Priyansh Bhatnagar, Zihan Xia, Nam Sung Kim, Tajana Rosing, Mingu Kang
https://arxiv.org/abs/2506.00020

Hybrid SLC-MLC RRAM Mixed-Signal Processing-in-Memory Architecture for Transformer Acceleration via Gradient Redistribution
Transformers, while revolutionary, face challenges due to their demanding computational cost and large data movement. To address this, we propose HyFlexPIM, a novel mixed-signal processing-in-memory (PIM) accelerator for inference that flexibly utilizes both single-level cell (SLC) and multi-level cell (MLC) RRAM technologies to trade-off accuracy and efficiency. HyFlexPIM achieves efficient dual-mode operation by utilizing digital PIM for high-precision and write-intensive operations while ana…

Identifying interactions across brain areas while accounting for individual-neuron dynamics with a Transformer-based variational autoencoder
Qi Xin, Robert E. Kass
https://arxiv.org/abs/2506.02263

Identifying interactions across brain areas while accounting for individual-neuron dynamics with a Transformer-based variational autoencoder
Advances in large-scale recording technologies now enable simultaneous measurements from multiple brain areas, offering new opportunities to study signal transmission across interacting components of neural circuits. However, neural responses exhibit substantial trial-to-trial variability, often driven by unobserved factors such as subtle changes in animal behavior or internal states. To prevent evolving background dynamics from contaminating identification of functional coupling, we developed …

Uncertainty-Aware Genomic Classification of Alzheimer's Disease: A Transformer-Based Ensemble Approach with Monte Carlo Dropout
Taeho Jo, Eun Hye Lee, Alzheimer's Disease Sequencing Project
https://arxiv.org/abs/2506.00662

Uncertainty-Aware Genomic Classification of Alzheimer's Disease: A Transformer-Based Ensemble Approach with Monte Carlo Dropout
INTRODUCTION: Alzheimer's disease (AD) is genetically complex, complicating robust classification from genomic data. METHODS: We developed a transformer-based ensemble model (TrUE-Net) using Monte Carlo Dropout for uncertainty estimation in AD classification from whole-genome sequencing (WGS). We combined a transformer that preserves single-nucleotide polymorphism (SNP) sequence structure with a concurrent random forest using flattened genotypes. An uncertainty threshold separated samples into …

Diffusion Transformer-based Universal Dose Denoising for Pencil Beam Scanning Proton Therapy
Yuzhen Ding, Jason Holmes, Hongying Feng, Martin Bues, Lisa A. McGee, Jean-Claude M. Rwigema, Nathan Y. Yu, Terence S. Sio, Sameer R. Keole, William W. Wong, Steven E. Schild, Jonathan B. Ashman, Sujay A. Vora, Daniel J. Ma, Samir H. Patel, Wei Liu
https://…

Diffusion Transformer-based Universal Dose Denoising for Pencil Beam Scanning Proton Therapy
Purpose: Intensity-modulated proton therapy (IMPT) offers precise tumor coverage while sparing organs at risk (OARs) in head and neck (H&N) cancer. However, its sensitivity to anatomical changes requires frequent adaptation through online adaptive radiation therapy (oART), which depends on fast, accurate dose calculation via Monte Carlo (MC) simulations. Reducing particle count accelerates MC but degrades accuracy. To address this, denoising low-statistics MC dose maps is proposed to enable fas…

This https://arxiv.org/abs/2502.13128 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSD_…

SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation
Text-to-song generation, the task of creating vocals and accompaniment from textual inputs, poses significant challenges due to domain complexity and data scarcity. Existing approaches often employ multi-stage generation procedures, leading to cumbersome training and inference pipelines, as well as suboptimal overall generation quality due to error accumulation across stages. In this paper, we propose SongGen, a fully open-source, single-stage auto-regressive transformer designed for controllab…

Transformative or Conservative? Conservation laws for ResNets and Transformers
Sibylle Marcotte, R\'emi Gribonval, Gabriel Peyr\'e
https://arxiv.org/abs/2506.06194

Transformative or Conservative? Conservation laws for ResNets and Transformers
While conservation laws in gradient flow training dynamics are well understood for (mostly shallow) ReLU and linear networks, their study remains largely unexplored for more practical architectures. This paper bridges this gap by deriving and analyzing conservation laws for modern architectures, with a focus on convolutional ResNets and Transformer networks. For this, we first show that basic building blocks such as ReLU (or linear) shallow networks, with or without convolution, have easily exp…

This https://arxiv.org/abs/2503.18617 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_…

Scaling Laws for Emulation of Stellar Spectra
Neural network-based emulators for the inference of stellar parameters and elemental abundances represent an increasingly popular methodology in modern spectroscopic surveys. However, these approaches are often constrained by their emulation precision and domain transfer capabilities. Greater generalizability has previously been achieved only with significantly larger model architectures, as demonstrated by Transformer-based models in natural language processing. This observation aligns with ne…

SNIFR : Boosting Fine-Grained Child Harmful Content Detection Through Audio-Visual Alignment with Cascaded Cross-Transformer
Orchid Chetia Phukan, Mohd Mujtaba Akhtar, Girish, Swarup Ranjan Behera, Abu Osama Siddiqui, Sarthak Jain, Priyabrata Mallick, Jaya Sai Kiran Patibandla, Pailla Balakrishna Reddy, Arun Balaji Buduru, Rajesh Sharma
https://

SNIFR : Boosting Fine-Grained Child Harmful Content Detection Through Audio-Visual Alignment with Cascaded Cross-Transformer
As video-sharing platforms have grown over the past decade, child viewership has surged, increasing the need for precise detection of harmful content like violence or explicit scenes. Malicious users exploit moderation systems by embedding unsafe content in minimal frames to evade detection. While prior research has focused on visual cues and advanced such fine-grained detection, audio features remain underexplored. In this study, we embed audio cues with visual for fine-grained child harmful c…

Tug-of-war between idiom's figurative and literal meanings in LLMs
Soyoung Oh, Xinting Huang, Mathis Pink, Michael Hahn, Vera Demberg
https://arxiv.org/abs/2506.01723

Tug-of-war between idiom's figurative and literal meanings in LLMs
Idioms present a unique challenge for language models due to their non-compositional figurative meanings, which often strongly diverge from the idiom's literal interpretation. This duality requires a model to learn representing and deciding between the two meanings to interpret an idiom in a figurative sense, or literally. In this paper, we employ tools from mechanistic interpretability to trace how a large pretrained causal transformer (LLama3.2-1B-base) deals with this ambiguity. We localize …

Discharge dynamics in a cylindrical SDBD prototype reactor under ns-pulsed and sinusoidal AC operation
Konstantinos Giotis (HVL, ECE, LSPM), Dimitrios Stefas (LSPM), Yanis Agha (LSPM), Hans H\"oft (INP), Xavier Duten (LSPM), Panagiotis Svarnas (HVL, ECE), Guillaume Lombardi (LSPM), Kristaq Gazeli (LSPM)
https://arxiv.org/ab…

Discharge dynamics in a cylindrical SDBD prototype reactor under ns-pulsed and sinusoidal AC operation
We developed a prototype reactor generating surface dielectric barrier discharges (SDBDs) in ambient air, designed for consistent operation while preventing constructive material degradation. It features detachable stainless steel electrodes and quartz dielectric to ensure precise fabrication. The grounded electrode is fully immersed into transformer oil drastically suppressing undesired parasitic discharges. The device efficiently sustains ns-pulsed and AC discharges at 10 kHz, enabling fundam…

Multi-modal brain MRI synthesis based on SwinUNETR
Haowen Pang, Weiyan Guo, Chuyang Ye
https://arxiv.org/abs/2506.02467 https://arxiv…

Multi-modal brain MRI synthesis based on SwinUNETR
Multi-modal brain magnetic resonance imaging (MRI) plays a crucial role in clinical diagnostics by providing complementary information across different imaging modalities. However, a common challenge in clinical practice is missing MRI modalities. In this paper, we apply SwinUNETR to the synthesize of missing modalities in brain MRI. SwinUNETR is a novel neural network architecture designed for medical image analysis, integrating the strengths of Swin Transformer and convolutional neural networ…

This https://arxiv.org/abs/2505.17452 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_…

Deblending Overlapping Galaxies in DECaLS Using Transformer-Based Algorithm: A Method Combining Multiple Bands and Data Types
In large-scale galaxy surveys, particularly deep ground-based photometric studies, galaxy blending is inevitable and poses a potential primary systematic uncertainty for upcoming surveys. Current deblenders predominantly rely on analytical modeling of galaxy profiles, facing limitations due to inflexible and imprecise models. We present a novel approach using a U-net structured transformer-based network for deblending astronomical images, which we term the CAT-deblender. It was trained using bo…

This https://arxiv.org/abs/2503.02407 has been replaced.
initial toot: https://mastoxiv.page/@a…

Wyckoff Transformer: Generation of Symmetric Crystals
Crystal symmetry plays a fundamental role in determining its physical, chemical, and electronic properties such as electrical and thermal conductivity, optical and polarization behavior, and mechanical strength. Almost all known crystalline materials have internal symmetry. However, this is often inadequately addressed by existing generative models, making the consistent generation of stable and symmetrically valid crystal structures a significant challenge. We introduce WyFormer, a generative …

Controllable Text-to-Speech Synthesis with Masked-Autoencoded Style-Rich Representation
Yongqi Wang, Chunlei Zhang, Hangting Chen, Zhou Zhao, Dong Yu
https://arxiv.org/abs/2506.02997

Controllable Text-to-Speech Synthesis with Masked-Autoencoded Style-Rich Representation
Controllable TTS models with natural language prompts often lack the ability for fine-grained control and face a scarcity of high-quality data. We propose a two-stage style-controllable TTS system with language models, utilizing a quantized masked-autoencoded style-rich representation as an intermediary. In the first stage, an autoregressive transformer is used for the conditional generation of these style-rich tokens from text and control signals. The second stage generates codec tokens from b…

This https://arxiv.org/abs/2502.17327 has been replaced.
link: https://scholar.google.com/scholar?q=a

AnyTop: Character Animation Diffusion with Any Topology
Generating motion for arbitrary skeletons is a longstanding challenge in computer graphics, remaining largely unexplored due to the scarcity of diverse datasets and the irregular nature of the data. In this work, we introduce AnyTop, a diffusion model that generates motions for diverse characters with distinct motion dynamics, using only their skeletal structure as input. Our work features a transformer-based denoising network, tailored for arbitrary skeleton learning, integrating topology info…

This https://arxiv.org/abs/2501.18564 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csRO_…

SAM2Act: Integrating Visual Foundation Model with A Memory Architecture for Robotic Manipulation
Robotic manipulation systems operating in diverse, dynamic environments must exhibit three critical abilities: multitask interaction, generalization to unseen scenarios, and spatial memory. While significant progress has been made in robotic manipulation, existing approaches often fall short in generalization to complex environmental variations and addressing memory-dependent tasks. To bridge this gap, we introduce SAM2Act, a multi-view robotic transformer-based policy that leverages multi-reso…

This https://arxiv.org/abs/2409.03833 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_grqc_…

Sequence modeling of higher-order wave modes of binary black hole mergers
Higher-order gravitational wave modes from quasi-circular, spinning, non-precessing binary black hole mergers encode key information about these systems' nonlinear dynamics. We model these waveforms using transformer architectures, targeting the evolution from late inspiral through ringdown. Our data is derived from the \texttt{NRHybSur3dq8} surrogate model, which includes spherical harmonic modes up to $\ell \leq 4$ (excluding $(4,0)$, $(4,\pm1)$ and including $(5,5)$ modes). These waveforms s…

Trading Under Uncertainty: A Distribution-Based Strategy for Futures Markets Using FutureQuant Transformer
Wenhao Guo, Yuda Wang, Zeqiao Huang, Changjiang Zhang, Shumin ma
https://arxiv.org/abs/2505.05595

Trading Under Uncertainty: A Distribution-Based Strategy for Futures Markets Using FutureQuant Transformer
In the complex landscape of traditional futures trading, where vast data and variables like real-time Limit Order Books (LOB) complicate price predictions, we introduce the FutureQuant Transformer model, leveraging attention mechanisms to navigate these challenges. Unlike conventional models focused on point predictions, the FutureQuant model excels in forecasting the range and volatility of future prices, thus offering richer insights for trading strategies. Its ability to parse and learn from…

Generalizable, real-time neural decoding with hybrid state-space models
Avery Hee-Woon Ryoo, Nanda H. Krishna, Ximeng Mao, Mehdi Azabou, Eva L. Dyer, Matthew G. Perich, Guillaume Lajoie
https://arxiv.org/abs/2506.05320

Generalizable, real-time neural decoding with hybrid state-space models
Real-time decoding of neural activity is central to neuroscience and neurotechnology applications, from closed-loop experiments to brain-computer interfaces, where models are subject to strict latency constraints. Traditional methods, including simple recurrent neural networks, are fast and lightweight but often struggle to generalize to unseen data. In contrast, recent Transformer-based approaches leverage large-scale pretraining for strong generalization performance, but typically have much l…

KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction
Jang-Hyun Kim, Jinuk Kim, Sangwoo Kwon, Jae W. Lee, Sangdoo Yun, Hyun Oh Song
https://arxiv.org/abs/2505.23416

KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction
Transformer-based large language models (LLMs) cache context as key-value (KV) pairs during inference. As context length grows, KV cache sizes expand, leading to substantial memory overhead and increased attention latency. This paper introduces KVzip, a query-agnostic KV cache eviction method enabling effective reuse of compressed KV caches across diverse queries. KVzip quantifies the importance of a KV pair using the underlying LLM to reconstruct original contexts from cached KV pairs, subsequ…

This https://arxiv.org/abs/2504.19605 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

A Comparative Study on Positional Encoding for Time-frequency Domain Dual-path Transformer-based Source Separation Models
In this study, we investigate the impact of positional encoding (PE) on source separation performance and the generalization ability to long sequences (length extrapolation) in Transformer-based time-frequency (TF) domain dual-path models. The length extrapolation capability in TF-domain dual-path models is a crucial factor, as it affects not only their performance on long-duration inputs but also their generalizability to signals with unseen sampling rates. While PE is known to significantly i…

Sleep Brain and Cardiac Activity Predict Cognitive Flexibility and Conceptual Reasoning Using Deep Learning
Boshra Khajehpiri, Eric Granger, Massimiliano de Zambotti, Fiona C. Baker, Mohamad Forouzanfar
https://arxiv.org/abs/2506.00279

Sleep Brain and Cardiac Activity Predict Cognitive Flexibility and Conceptual Reasoning Using Deep Learning
Despite extensive research on the relationship between sleep and cognition, the connection between sleep microstructure and human performance across specific cognitive domains remains underexplored. This study investigates whether deep learning models can predict executive functions, particularly cognitive adaptability and conceptual reasoning from physiological processes during a night's sleep. To address this, we introduce CogPSGFormer, a multi-scale convolutional-transformer model designed t…

BD at BEA 2025 Shared Task: MPNet Ensembles for Pedagogical Mistake Identification and Localization in AI Tutor Responses
Shadman Rohan, Ishita Sur Apan, Muhtasim Ibteda Shochcho, Md Fahim, Mohammad Ashfaq Ur Rahman, AKM Mahbubur Rahman, Amin Ahsan Ali
https://arxiv.org/abs/2506.01817

BD at BEA 2025 Shared Task: MPNet Ensembles for Pedagogical Mistake Identification and Localization in AI Tutor Responses
We present Team BD's submission to the BEA 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutors, under Track 1 (Mistake Identification) and Track 2 (Mistake Location). Both tracks involve three-class classification of tutor responses in educational dialogues - determining if a tutor correctly recognizes a student's mistake (Track 1) and whether the tutor pinpoints the mistake's location (Track 2). Our system is built on MPNet, a Transformer-based language model that combines …

This https://arxiv.org/abs/2505.20156 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCV_…

HunyuanVideo-Avatar: High-Fidelity Audio-Driven Human Animation for Multiple Characters
Recent years have witnessed significant progress in audio-driven human animation. However, critical challenges remain in (i) generating highly dynamic videos while preserving character consistency, (ii) achieving precise emotion alignment between characters and audio, and (iii) enabling multi-character audio-driven animation. To address these challenges, we propose HunyuanVideo-Avatar, a multimodal diffusion transformer (MM-DiT)-based model capable of simultaneously generating dynamic, emotion-…

I have released LLama2.c64 - an LLM running on a C64 with 2MB REU. It runs the Llama2 LLM architecture, using the tokenizer and weights from the Tinystories 260K model.
It's a storytelling model that tries its best to spin your prompt into a story, as if told by a kindergarten child. It will generate one output token about every 8 minutes.
…

This https://arxiv.org/abs/2505.14314 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csAR_…

Low-Cost FlashAttention with Fused Exponential and Multiplication Hardware Operators
Attention mechanisms, particularly within Transformer architectures and large language models (LLMs), have revolutionized sequence modeling in machine learning and artificial intelligence applications. To compute attention for increasingly long sequences, specialized accelerators have been proposed to execute key attention steps directly in hardware. Among the various recently proposed architectures, those based on variants of the FlashAttention algorithm, originally designed for GPUs, stand ou…

Prospects and challenges of Bluetooth backscatters system
Jingyun Du
https://arxiv.org/abs/2505.21526 https://arxiv.org/pdf/2505.2152…

Prospects and challenges of Bluetooth backscatters system
Bluetooth backscatter systems, as a crucial technology for low-power communication in the Internet of Things (IoT), have witnessed remarkable development in recent years. This article comprehensively analyzes multiple related papers, including the latest advancements in RF-Transformer and B2Loc systems, summarizes their research progress, challenges faced, and classification, and explores the application prospects and future development directions of such systems. Bluetooth backscatter systems …

This https://arxiv.org/abs/2504.01195 has been replaced.
initial toot: https://mastoxiv.page/@ar…

Towards Signed Distance Function based Metamaterial Design: Neural Operator Transformer for Forward Prediction and Diffusion Model for Inverse Design
The inverse design of metamaterial architectures presents a significant challenge, particularly for nonlinear mechanical properties involving large deformations, buckling, contact, and plasticity. Traditional methods, such as gradient-based optimization, and recent generative deep-learning approaches often rely on binary pixel-based representations, which introduce jagged edges that hinder finite element (FE) simulations and 3D printing. To overcome these challenges, we propose an inverse desig…

This https://arxiv.org/abs/2504.06212 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

NNN: Next-Generation Neural Networks for Marketing Measurement
We present NNN, an experimental Transformer-based neural network approach to marketing measurement. Unlike Marketing Mix Models (MMMs) which rely on scalar inputs and parametric decay functions, NNN uses rich embeddings to capture both quantitative and qualitative aspects of marketing and organic channels (e.g., search queries, ad creatives). This, combined with its attention mechanism, potentially enables NNN to model complex interactions, capture long-term effects, and improve sales attributi…

CF-DETR: Coarse-to-Fine Transformer for Real-Time Object Detection
Woojin Shin, Donghwa Kang, Byeongyun Park, Brent Byunghoon Kang, Jinkyu Lee, Hyeongboo Baek
https://arxiv.org/abs/2505.23317

CF-DETR: Coarse-to-Fine Transformer for Real-Time Object Detection
Detection Transformers (DETR) are increasingly adopted in autonomous vehicle (AV) perception systems due to their superior accuracy over convolutional networks. However, concurrently executing multiple DETR tasks presents significant challenges in meeting firm real-time deadlines (R1) and high accuracy requirements (R2), particularly for safety-critical objects, while navigating the inherent latency-accuracy trade-off under resource constraints. Existing real-time DNN scheduling approaches ofte…

MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity Reconstruction and Generation
Yakun Song, Jiawei Chen, Xiaobin Zhuang, Chenpeng Du, Ziyang Ma, Jian Wu, Jian Cong, Dongya Jia, Zhuo Chen, Yuping Wang, Yuxuan Wang, Xie Chen
https://arxiv.org/abs/2506.00385

MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity Reconstruction and Generation
Neural audio codecs have made significant strides in efficiently mapping raw audio waveforms into discrete token representations, which are foundational for contemporary audio generative models. However, most existing codecs are optimized primarily for reconstruction quality, often at the expense of the downstream modelability of the encoded tokens. Motivated by the need to overcome this bottleneck, we introduce $\textbf{MagiCodec}$, a novel single-layer, streaming Transformer-based audio codec…

This https://arxiv.org/abs/2410.09795 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_qbi…

WGFormer: An SE(3)-Transformer Driven by Wasserstein Gradient Flows for Molecular Ground-State Conformation Prediction
Predicting molecular ground-state conformation (i.e., energy-minimized conformation) is crucial for many chemical applications such as molecular docking and property prediction. Classic energy-based simulation is time-consuming when solving this problem, while existing learning-based methods have advantages in computational efficiency but sacrifice accuracy and interpretability. In this work, we propose a novel and effective method to bridge the energy-based simulation and the learning-based st…

This https://arxiv.org/abs/2504.06354 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_qbi…

Multihead self-attention in cortico-thalamic circuits
Both biological cortico-thalamic networks and artificial transformer networks use canonical computations to perform a wide range of cognitive tasks. In this work, we propose that the structure of cortico-thalamic circuits is well suited to realize a computation analogous to multihead self-attention, the main algorithmic innovation of transformers. We start with the concept of a cortical unit module or microcolumn, and propose that superficial and deep pyramidal cells carry distinct computationa…

This https://arxiv.org/abs/2410.21295 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_qbi…

TRENDY: Gene Regulatory Network Inference Enhanced by Transformer
Gene regulatory networks (GRNs) play a crucial role in the control of cellular functions. Numerous methods have been developed to infer GRNs from gene expression data, including mechanism-based approaches, information-based approaches, and more recent deep learning techniques, the last of which often overlook the underlying gene expression mechanisms. In this work, we introduce TRENDY, a novel GRN inference method that integrates transformer models to enhance the mechanism-based WENDY approach.…

This https://arxiv.org/abs/2505.23527 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

Normalizing Flows are Capable Models for RL
Modern reinforcement learning (RL) algorithms have found success by using powerful probabilistic models, such as transformers, energy-based models, and diffusion/flow-based models. To this end, RL researchers often choose to pay the price of accommodating these models into their algorithms -- diffusion models are expressive, but are computationally intensive due to their reliance on solving differential equations, while autoregressive transformer models are scalable but typically require learni…

RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with Global Illumination
Chong Zeng, Yue Dong, Pieter Peers, Hongzhi Wu, Xin Tong
https://arxiv.org/abs/2505.21925

RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with Global Illumination
We present RenderFormer, a neural rendering pipeline that directly renders an image from a triangle-based representation of a scene with full global illumination effects and that does not require per-scene training or fine-tuning. Instead of taking a physics-centric approach to rendering, we formulate rendering as a sequence-to-sequence transformation where a sequence of tokens representing triangles with reflectance properties is converted to a sequence of output tokens representing small patc…

Phi-Omni-ST: A multimodal language model for direct speech-to-speech translation
Yuxuan Hu, Haibin Wu, Ruchao Fan, Xiaofei Wang, Heng Lu, Yao Qian, Jinyu Li
https://arxiv.org/abs/2506.04392

Phi-Omni-ST: A multimodal language model for direct speech-to-speech translation
Speech-aware language models (LMs) have demonstrated capabilities in understanding spoken language while generating text-based responses. However, enabling them to produce speech output efficiently and effectively remains a challenge. In this paper, we present Phi-Omni-ST, a multimodal LM for direct speech-to-speech translation (ST), built on the open-source Phi-4 MM model. Phi-Omni-ST extends its predecessor by generating translated speech using an audio transformer head that predicts audio to…

This https://arxiv.org/abs/2502.19679 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDL_…

Architectural Vulnerability and Reliability Challenges in AI Text Annotation: A Survey-Inspired Framework with Independent Probability Assessment
Large Language Models, despite their power, have a fundamental architectural vulnerability stemming from their causal transformer design -- order sensitivity. This architectural constraint may distorts classification outcomes when prompt elements like label options are reordered, revealing a theoretical gap between accuracy metrics and true model reliability. The paper conceptualizes this vulnerability through the lens of survey methodology, where respondent biases parallel LLM positional depen…

This https://arxiv.org/abs/2505.17662 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

Automating Versatile Time-Series Analysis with Tiny Transformers on Embedded FPGAs
Transformer-based models have shown strong performance across diverse time-series tasks, but their deployment on resource-constrained devices remains challenging due to high memory and computational demand. While prior work targeting Microcontroller Units (MCUs) has explored hardware-specific optimizations, such approaches are often task-specific and limited to 8-bit fixed-point precision. Field-Programmable Gate Arrays (FPGAs) offer greater flexibility, enabling fine-grained control over data …

Refining Datapath for Microscaling ViTs
Can Xiao, Jianyi Cheng, Aaron Zhao
https://arxiv.org/abs/2505.22194 https://arxiv.org/pdf/250…

Refining Datapath for Microscaling ViTs
Vision Transformers (ViTs) leverage the transformer architecture to effectively capture global context, demonstrating strong performance in computer vision tasks. A major challenge in ViT hardware acceleration is that the model family contains complex arithmetic operations that are sensitive to model accuracy, such as the Softmax and LayerNorm operations, which cannot be mapped onto efficient hardware with low precision. Existing methods only exploit parallelism in the matrix multiplication ope…

This https://arxiv.org/abs/2505.20896 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

How Do Transformers Learn Variable Binding in Symbolic Programs?
Variable binding -- the ability to associate variables with values -- is fundamental to symbolic computation and cognition. Although classical architectures typically implement variable binding via addressable memory, it is not well understood how modern neural networks lacking built-in binding operations may acquire this capacity. We investigate this by training a Transformer to dereference queried variables in symbolic programs where variables are assigned either numerical constants or other …

This https://arxiv.org/abs/2503.09605 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_qbi…

QWENDY: Gene Regulatory Network Inference Enhanced by Large Language Model and Transformer
Knowing gene regulatory networks (GRNs) is important for understanding various biological mechanisms. In this paper, we present a method, QWENDY, that uses single-cell gene expression data measured at four time points to infer GRNs. Based on a linear gene expression model, it solves the transformation for the covariance matrices. Unlike its predecessor WENDY, QWENDY avoids solving a non-convex optimization problem and produces a unique solution. Then we enhance QWENDY by the transformer neural …

DeepPlantCRE: A Transformer-CNN Hybrid Framework for Plant Gene Expression Modeling and Cross-Species Generalization
Yingjun Wu, Jingyun Huang, Liang Ming, Pengcheng Deng, Maojun Wang, Zeyu Zhang
https://arxiv.org/abs/2505.09883

DeepPlantCRE: A Transformer-CNN Hybrid Framework for Plant Gene Expression Modeling and Cross-Species Generalization
The investigation of plant transcriptional regulation constitutes a fundamental basis for crop breeding, where cis-regulatory elements (CREs), as the key factor determining gene expression, have become the focus of crop genetic improvement research. Deep learning techniques, leveraging their exceptional capacity for high-dimensional feature extraction and nonlinear regulatory relationship modeling, have been extensively employed in this field. However, current methodologies present notable limi…

This https://arxiv.org/abs/2410.12675 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

AttentiveMOS: A Lightweight Attention-Only Model for Speech Quality Prediction
Research in modeling subjective metrics for quality assessment has led to the development of no-reference speech models that directly operate on utterance waveforms to predict the Mean Opinion Score (MOS). These models often rely on convolutional layers for local feature extraction and embeddings from impractically large pretrained networks to enhance generalization. We propose an attention-only model based on Swin transformer and standard transformer layers to extract local context features an…

This https://arxiv.org/abs/2505.23527 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

Normalizing Flows are Capable Models for RL
Modern reinforcement learning (RL) algorithms have found success by using powerful probabilistic models, such as transformers, energy-based models, and diffusion/flow-based models. To this end, RL researchers often choose to pay the price of accommodating these models into their algorithms -- diffusion models are expressive, but are computationally intensive due to their reliance on solving differential equations, while autoregressive transformer models are scalable but typically require learni…

Transformers as Multi-task Learners: Decoupling Features in Hidden Markov Models
Yifan Hao, Chenlu Ye, Chi Han, Tong Zhang
https://arxiv.org/abs/2506.01919

Transformers as Multi-task Learners: Decoupling Features in Hidden Markov Models
Transformer based models have shown remarkable capabilities in sequence learning across a wide range of tasks, often performing well on specific task by leveraging input-output examples. Despite their empirical success, a comprehensive theoretical understanding of this phenomenon remains limited. In this work, we investigate the layerwise behavior of Transformers to uncover the mechanisms underlying their multi-task generalization ability. Taking explorations on a typical sequence model, i.e, H…

This https://arxiv.org/abs/2505.23378 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

Meta-Learning Approaches for Speaker-Dependent Voice Fatigue Models
Speaker-dependent modelling can substantially improve performance in speech-based health monitoring applications. While mixed-effect models are commonly used for such speaker adaptation, they require computationally expensive retraining for each new observation, making them impractical in a production environment. We reformulate this task as a meta-learning problem and explore three approaches of increasing complexity: ensemble-based distance models, prototypical networks, and transformer-based…

Automatic detection of abnormal clinical EEG: comparison of a finetuned foundation model with two deep learning models
Aurore Bussalb, Fran\c{c}ois Le Gac, Guillaume Jubien, Mohamed Rahmouni, Ruggero G. Bettinardi, Pedro Marinho R. de Oliveira, Phillipe Derambure, Nicolas Gaspard, Jacques Jonas, Louis Maillard, Laurent Vercueil, Herv\'e Vespignani, Philippe Laval, Laurent Koessler, Ulysse Gimenez

Automatic detection of abnormal clinical EEG: comparison of a finetuned foundation model with two deep learning models
Electroencephalography (EEG) is commonly used by physicians for the diagnosis of numerous neurological disorders. Due to the large volume of EEGs requiring interpretation and the specific expertise involved, artificial intelligence-based tools are being developed to assist in their visual analysis. In this paper, we compare two deep learning models (CNN-LSTM and Transformer-based) with BioSerenity-E1, a recently proposed foundation model, in the task of classifying entire EEG recordings as norm…

Leveraging AM and FM Rhythm Spectrograms for Dementia Classification and Assessment
Parismita Gogoi, Vishwanath Pratap Singh, Seema Khadirnaikar, Soma Siddhartha, Sishir Kalita, Jagabandhu Mishra, Md Sahidullah, Priyankoo Sarmah, S. R. M. Prasanna
https://arxiv.org/abs/2506.00861

Leveraging AM and FM Rhythm Spectrograms for Dementia Classification and Assessment
This study explores the potential of Rhythm Formant Analysis (RFA) to capture long-term temporal modulations in dementia speech. Specifically, we introduce RFA-derived rhythm spectrograms as novel features for dementia classification and regression tasks. We propose two methodologies: (1) handcrafted features derived from rhythm spectrograms, and (2) a data-driven fusion approach, integrating proposed RFA-derived rhythm spectrograms with vision transformer (ViT) for acoustic representations alo…

Correlating instruction-tuning (in multimodal models) with vision-language processing (in the brain)
Subba Reddy Oota, Akshett Jindal, Ishani Mondal, Khushbu Pahwa, Satya Sai Srinath Namburi, Manish Shrivastava, Maneesh Singh, Bapi S. Raju, Manish Gupta
https://arxiv.org/abs/2505.20029

Correlating instruction-tuning (in multimodal models) with vision-language processing (in the brain)
Transformer-based language models, though not explicitly trained to mimic brain recordings, have demonstrated surprising alignment with brain activity. Progress in these models-through increased size, instruction-tuning, and multimodality-has led to better representational alignment with neural data. Recently, a new class of instruction-tuned multimodal LLMs (MLLMs) have emerged, showing remarkable zero-shot capabilities in open-ended multimodal vision tasks. However, it is unknown whether MLLM…

This https://arxiv.org/abs/2505.13814 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Articulatory Feature Prediction from Surface EMG during Speech Production
We present a model for predicting articulatory features from surface electromyography (EMG) signals during speech production. The proposed model integrates convolutional layers and a Transformer block, followed by separate predictors for articulatory features. Our approach achieves a high prediction correlation of approximately 0.9 for most articulatory features. Furthermore, we demonstrate that these predicted articulatory features can be decoded into intelligible speech waveforms. To our know…