Tootfinder

Optimizing Neural Networks with Learnable Non-Linear Activation Functions via Lookup-Based FPGA Acceleration
Mengyuan Yin, Benjamin Chen Ming Choong, Chuping Qu, Rick Siow Mong Goh, Weng-Fai Wong, Tao Luo
https://arxiv.org/abs/2508.17069

Optimizing Neural Networks with Learnable Non-Linear Activation Functions via Lookup-Based FPGA Acceleration
Learned activation functions in models like Kolmogorov-Arnold Networks (KANs) outperform fixed-activation architectures in terms of accuracy and interpretability; however, their computational complexity poses critical challenges for energy-constrained edge AI deployments. Conventional CPUs/GPUs incur prohibitive latency and power costs when evaluating higher order activations, limiting deployability under ultra-tight energy budgets. We address this via a reconfigurable lookup architecture with …

Exploiting Vocabulary Frequency Imbalance in Language Model Pre-training
Woojin Chung, Jeonghoon Kim
https://arxiv.org/abs/2508.15390 https://arxiv.org/pdf…

Exploiting Vocabulary Frequency Imbalance in Language Model Pre-training
Large language models are trained with tokenizers, and the resulting token distribution is highly imbalanced: a few words dominate the stream while most occur rarely. Recent practice favors ever-larger vocabularies, but the source of the benefit is unclear. We conduct a controlled study that scales the language model's vocabulary from 24K to 196K while holding data, compute, and optimization fixed. We first quantify the complexity of tokenized text, formalized via Kolmogorov complexity, and sho…

Bridging Kolmogorov Complexity and Deep Learning: Asymptotically Optimal Description Length Objectives for Transformers
Peter Shaw, James Cohan, Jacob Eisenstein, Kristina Toutanova
https://arxiv.org/abs/2509.22445

Bridging Kolmogorov Complexity and Deep Learning: Asymptotically Optimal Description Length Objectives for Transformers
The Minimum Description Length (MDL) principle offers a formal framework for applying Occam's razor in machine learning. However, its application to neural networks such as Transformers is challenging due to the lack of a principled, universal measure for model complexity. This paper introduces the theoretical notion of asymptotically optimal description length objectives, grounded in the theory of Kolmogorov complexity. We establish that a minimizer of such an objective achieves optimal compre…

Characterizing Agent-Based Model Dynamics via $\epsilon$-Machines and Kolmogorov-Style Complexity
Roberto Garrone (University of Milano-Bicocca)
https://arxiv.org/abs/2510.12729

Characterizing Agent-Based Model Dynamics via $ε$-Machines and Kolmogorov-Style Complexity
We propose a two-level information-theoretic framework for characterizing the informational organization of Agent-Based Model (ABM) dynamics within the broader paradigm of Complex Adaptive Systems (CAS). At the macro level, a pooled $ε$-machine is reconstructed as a reference model that summarizes the system-wide informational regime. At the micro level, $ε$-machines are reconstructed for each caregiver-elder dyad and variable, and are complemented with algorithm-agnostic Kolmogorov-style mea…

Quantum circuit complexity and unsupervised machine learning of topological order
Yanming Che, Clemens Gneiting, Xiaoguang Wang, Franco Nori
https://arxiv.org/abs/2508.04486 htt…

Quantum circuit complexity and unsupervised machine learning of topological order
Inspired by the close relationship between Kolmogorov complexity and unsupervised machine learning, we explore quantum circuit complexity, an important concept in quantum computation and quantum information science, as a pivot to understand and to build interpretable and efficient unsupervised machine learning for topological order in quantum many-body systems. To span a bridge from conceptual power to practical applicability, we present two theorems that connect Nielsen's quantum circuit compl…

Predictable Compression Failures: Why Language Models Actually Hallucinate
Leon Chlon, Ahmed Karim, Maggie Chlon
https://arxiv.org/abs/2509.11208 https://a…

Predictable Compression Failures: Why Language Models Actually Hallucinate
Large language models perform near-Bayesian inference yet violate permutation invariance on exchangeable data. We resolve this by showing transformers minimize expected conditional description length (cross-entropy) over orderings, $\mathbb{E}_π[\ell(Y \mid Γ_π(X))]$, which admits a Kolmogorov-complexity interpretation up to additive constants, rather than the permutation-invariant description length $\ell(Y \mid X)$. This makes them Bayesian in expectation, not in realization. We derive (i)…

Event-based complexity in atmospheric turbulence
Paolo Paradisi, Rita Cesari
https://arxiv.org/abs/2509.24576 https://arxiv.org/pdf/2509.24576

Event-based complexity in atmospheric turbulence
Since the studies of Kolmogorov and Oboukhov in 1941, the problem of intermittent large velocity excursions was recognized to be one of the most intriguing and elusive aspects of turbulent flows. While many efforts were devoted since 1960s to the magnitude intermittency, related to the statistics of the large increments in the turbulent signals, the attention towards the so-called clustering intermittency has started to increase only in the last two decades, even if some pioneering studies were…

Formal Framework for Quantum Advantage
Harry Buhrman, Niklas Galke, Konstantinos Meichanetzidis
https://arxiv.org/abs/2510.01953 https://arxiv.org/pdf/2510…

Formal Framework for Quantum Advantage
Motivated by notions of quantum heuristics and by average-case rather than worst-case algorithmic analysis, we define quantum computational advantage in terms of individual problem instances. Inspired by the classical notions of Kolmogorov complexity and instance complexity, we define their quantum versions. This allows us to define queasy instances of computational problems, like e.g. Satisfiability and Factoring, as those whose quantum instance complexity is significantly smaller than their c…

Algorithmic Information Bounds for Distances and Orthogonal Projections
Peter Cholak, Marianna Cs\"ornyei, Neil Lutz, Patrick Lutz, Elvira Mayordomo, D. M. Stull
https://arxiv.org/abs/2509.05211

Algorithmic Information Bounds for Distances and Orthogonal Projections
We develop quantitative algorithmic information bounds for orthogonal projections and distances in the plane. Under mild independence conditions, the distance $|x-y|$ and a projection coordinate $p_e x$ each retain at least half the algorithmic information content of $x$ in the sense of finite-precision Kolmogorov complexity, up to lower-order terms. Our bounds support conditioning on coarser approximations, enabling case analyses across precision scales. The proofs introduce a surrog…

Strong Kurtz Randomness and Binary Expansions of Reordered Computable Numbers
Peter Hertling, Philip Janicki
https://arxiv.org/abs/2509.22406 https://arxiv…

Strong Kurtz Randomness and Binary Expansions of Reordered Computable Numbers
A real number is called left-computable if there exists a computable increasing sequence of rational numbers converging to it. In this article we investigate the Kolmogorov complexity and the binary expansions of a very specific subset of the left-computable numbers. We show in our main result that a real number is reordered computable if, and only if, it is left-computable and not strongly Kurtz random. In preparation of this, we characterize strong Kurtz randomness by a suitable notion of ran…

Properties of Algorithmic Information Distance
Marcus Hutter
https://arxiv.org/abs/2507.21988 https://arxiv.org/pdf/2507.21988

Properties of Algorithmic Information Distance
The domain-independent universal Normalized Information Distance based on Kolmogorov complexity has been (in approximate form) successfully applied to a variety of difficult clustering problems. In this paper we investigate theoretical properties of the un-normalized algorithmic information distance $d_K$. The main question we are asking in this work is what properties this curious distance has, besides being a metric. We show that many (in)finite-dimensional spaces can(not) be isometrically sc…