Tootfinder

Markov matrix perturbations to optimize dynamical and entropy functionals
Manuel Santos Gutierrez, Niccolo Zagli, Giulia Carigi
https://arxiv.org/abs/2507.14040

Markov matrix perturbations to optimize dynamical and entropy functionals
An important problem in applied dynamical systems is to compute the external forcing that provokes the largest response of a desired observable quantity. For this, we investigate the perturbation theory of Markov matrices in connection with linear response theory in statistical physics. We use perturbative expansions to derive linear algorithms to optimize physically relevant quantities such as: entropy, Kullback-Liebler-divergence and entropy production of Markov matrices and their related pro…

Long run control of nonhomogeneous Markov processes
{\L}ukasz Stettner
https://arxiv.org/abs/2506.15542 https://arxiv.org/pdf/2506.15…

Long run control of nonhomogeneous Markov processes
In the paper average reward per unit time and average risk sensitive reward functionals are considered for controlled nonhomogeneous Markov processes. Existence of solutions to suitable Bellman equations is shown. Continuity of the value functions with respect to risk parameter is also proved. Finally stability of functionals with respect to pointwise convergence of Markov controls is studied.

Towards Robust Causal Effect Identification Beyond Markov Equivalence
Kai Z. Teh, Kayvan Sadeghi, Terry Soo
https://arxiv.org/abs/2506.15561 https://

Towards Robust Causal Effect Identification Beyond Markov Equivalence
Causal effect identification typically requires a fully specified causal graph, which can be difficult to obtain in practice. We provide a sufficient criterion for identifying causal effects from a candidate set of Markov equivalence classes with added background knowledge, which represents cases where determining the causal graph up to a single Markov equivalence class is challenging. Such cases can happen, for example, when the untestable assumptions (e.g. faithfulness) that underlie causal d…

Markov dualities via the spectral decompositions of the two Markov generators in their bi-orthogonal basis of right and left eigenvectors
Cecile Monthus
https://arxiv.org/abs/2507.11041

Markov dualities via the spectral decompositions of the two Markov generators in their bi-orthogonal basis of right and left eigenvectors
The notion of Markov duality between two Markov processes that can live in two different configurations spaces $(x,{\tilde x})$ is revisited via the spectral decompositions of the two Markov generators in their bi-orthogonal basis of right and left eigenvectors. In this formulation, the two generators should have the same eigenvalues $(-E)$ that may be complex, while the duality function $Ω(x,{\tilde x})$ can be considered as a mapping between the right and the left eigenvectors of the two mod…

Multi-Component VAE with Gaussian Markov Random Field
Fouad Oubari, Mohamed El-Baha, Raphael Meunier, Rodrigue D\'ecatoire, Mathilde Mougeot
https://arxiv.org/abs/2507.12165

Multi-Component VAE with Gaussian Markov Random Field
Multi-component datasets with intricate dependencies, like industrial assemblies or multi-modal imaging, challenge current generative modeling techniques. Existing Multi-component Variational AutoEncoders typically rely on simplified aggregation strategies, neglecting critical nuances and consequently compromising structural coherence across generated components. To explicitly address this gap, we introduce the Gaussian Markov Random Field Multi-Component Variational AutoEncoder , a novel gener…

In our #ISE2025 lecture last Wednesday, we learned how in n-gram language models via Markov assumption and maximum likelihood estimation we can predict the probability of the occurrence of a word given a specific context (i.e. n words previous in the sequence of words).
#NLP

Finding the Smallest Possible Exact Aggregation of a Markov Chain
Patrick Sonnentag
https://arxiv.org/abs/2507.11157 https://arxiv.or…

Finding the Smallest Possible Exact Aggregation of a Markov Chain
Markov chains are an important tool for modelling and evaluating systems in computer science, economics, biology and numerous other fields. Thus, approximating Markov chains is a useful tool for decreasing the computational effort needed for analysis by reducing the size of the state space. So far, most approximative algorithms focused on finding approximations, which are, again, Markov chains. We remove this restriction and present an algorithm using the Arnoldi iteration. Further, we show tha…

Quantum Circuits for the Metropolis-Hastings Algorithm
Baptiste Claudon, Pablo Rodenas-Ruiz, Jean-Philip Piquemal, Pierre Monmarch\'e
https://arxiv.org/abs/2506.11576

Quantum Circuits for the Metropolis-Hastings Algorithm
Szegedy's quantization of a reversible Markov chain provides a quantum walk whose mixing time is quadratically smaller than that of the classical walk. Quantum computers are therefore expected to provide a speedup of Metropolis-Hastings (MH) simulations. Existing generic methods to implement the quantum walk require coherently computing the acceptance probabilities of the underlying Markov kernel. However, reversible computing methods require a number of qubits that scales with the complexity o…

Adapted Measures for Markov Interval Maps
{\L}ukasz Krzywo\'n
https://arxiv.org/abs/2506.14971 https://arxiv.org/pdf/2506.14971…

Adapted Measures for Markov Interval Maps
Adapted invariant measures, such as the natural area measure (Liouville), have a central place in the development of ergodic theory for billiards. These measures ensure local Pesin charts can be constructed almost everywhere even in the nonuniformly hyperbolic setting. Recently, for Sinai billiards satisfying certain conditions, the unique measure of maximal entropy has been shown to be adapted. However, not all positive entropy measures are. To investigate the connection between entropy and ad…

Eigenvalue Bounds for Symmetric Markov Chains on Multislices With Applications
Prashanth Amireddy, Amik Raj Behera, Srikanth Srinivasan, Madhu Sudan
https://arxiv.org/abs/2507.10731

Eigenvalue Bounds for Symmetric Markov Chains on Multislices With Applications
We consider random walks on ``balanced multislices'' of any ``grid'' that respects the ``symmetries'' of the grid, and show that a broad class of such walks are good spectral expanders. (A grid is a set of points of the form $\mathcal{S}^n$ for finite $\mathcal{S}$, and a balanced multi-slice is the subset that contains an equal number of coordinates taking every value in $\mathcal{S}$. A walk respects symmetries if the probability of going from $u = (u_1,\ldots,u_n)$ to $v = (v_1,\ldots,v_n)$ …

anyone wanna help a girl create a markov bot of herself

Cluster algebraic interpretation of generalized Markov numbers and their matrixizations
Esther Banaian, Yasuaki Gyoda
https://arxiv.org/abs/2507.06900 http…

Cluster algebraic interpretation of generalized Markov numbers and their matrixizations
Markov numbers, i.e. positive integers appearing in solutions to $x^2 + y^2 + z^2 = 3xyz$, can be viewed as specializations of cluster variables. The second author and Matsushita gave a generalization of the Markov equation, $x^2 + y^2 + z^2 + k_1yz + k_2xz + k_3xy = (3+k_1+k_2+k_3)xyz$, whose solutions can be viewed as specializations of cluster variables in generalized cluster algebras. We give two families of matrices in $SL(2,\mathbb{Z}[x_1^\pm,x_2^\pm,x_3^\pm])$ associated to these cluster…

Chance and Mass Interpretations of Probabilities in Markov Decision Processes (Extended Version)
Yun Chen Tsai, Kittiphon Phalakarn, S. Akshay, Ichiro Hasuo
https://arxiv.org/abs/2506.10377

Chance and Mass Interpretations of Probabilities in Markov Decision Processes (Extended Version)
Markov decision processes (MDPs) are a popular model for decision-making in the presence of uncertainty. The conventional view of MDPs in verification treats them as state transformers with probabilities defined over sequences of states and with schedulers making random choices. An alternative view, especially well-suited for modeling dynamical systems, defines MDPs as distribution transformers with schedulers distributing probability masses. Our main contribution is a unified semantical framew…

A note on intrinsic topologies of groups
S. Bardyla, L. Elliott, J. D. Mitchell, Y. P\'eresse
https://arxiv.org/abs/2506.11500 https://

A note on intrinsic topologies of groups
We investigate topologies on groups which arise naturally from their algebraic structure, including the Frechét-Markov, Hausdorff-Markov, and various kinds of Zariski topologies. Answering a question by Dikranjan and Toller, we show that there exists a countable abelian group in which no bounded version of the Zariski topology coincides with the full Zariski topology. Complementing a recent result by Goffe and Greenfeld, we show that on any group with no algebraicity the semigroup Zariski topo…

Rademacher learning rates for iterated random functions
Nikola Sandri\'c
https://arxiv.org/abs/2506.13946 https://arxiv.org/pdf/2…

Rademacher learning rates for iterated random functions
Most existing literature on supervised machine learning assumes that the training dataset is drawn from an i.i.d. sample. However, many real-world problems exhibit temporal dependence and strong correlations between the marginal distributions of the data-generating process, suggesting that the i.i.d. assumption is often unrealistic. In such cases, models naturally include time-series processes with mixing properties, as well as irreducible and aperiodic ergodic Markov chains. Moreover, the lear…

This https://arxiv.org/abs/2504.01759 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Alpha-Beta HMM: Hidden Markov Model Filtering with Equal Exit Probabilities and a Step-Size Parameter
The hidden Markov model (HMM) provides a powerful framework for inference in time-varying environments, where the underlying state evolves according to a Markov chain. To address the optimal filtering problem in general dynamic settings, we propose the $αβ$-HMM algorithm, which simplifies the state transition model to a Markov chain with equal exit probabilities and introduces a step-size parameter to balance the influence of observational data and the model. By analyzing the algorithm's dyna…

Markov approximation for controlled Hawkes Jump-Diffusions with general kernels
Mahmoud Khabou, Mehdi Talbi
https://arxiv.org/abs/2507.11294 https://

Markov approximation for controlled Hawkes Jump-Diffusions with general kernels
We present a Markov approximation for jump-diffusions whose jump part consists in a Hawkes process with intensity driven by a general (possibly non-monotone) kernel. Under minimal integrability conditions, the kernel can be approximated by a linear combination of exponential functions. This implies that Hawkes jump-diffusions can be approximated with Markov jump-diffusions. We illustrate the usefulness of this approximation by applying it to a class of stochastic control problems.

Analysis of Langevin midpoint methods using an anticipative Girsanov theorem
Matthew S. Zhang
https://arxiv.org/abs/2507.12791 https://

Analysis of Langevin midpoint methods using an anticipative Girsanov theorem
We introduce a new method for analyzing midpoint discretizations of stochastic differential equations (SDEs), which are frequently used in Markov chain Monte Carlo (MCMC) methods for sampling from a target measure $π\propto \exp(-V)$. Borrowing techniques from Malliavin calculus, we compute estimates for the Radon-Nikodym derivative for processes on $L^2([0, T); \mathbb{R}^d)$ which may anticipate the Brownian motion, in the sense that they may not be adapted to the filtration at the same time…

Anomalous diffusion for mass transport phenomena II: Subdiffusion in polydimethylsiloxane (PDMS)
Nathaniel G. Hermann, Dmitry A. Markov, M. Shane Hutson
https://arxiv.org/abs/2506.14600

Anomalous diffusion for mass transport phenomena II: Subdiffusion in polydimethylsiloxane (PDMS)
Polydimethylsiloxane (PDMS) is a glassy polymer widely used in biomedical engineering, namely in microfluidics applications. However, PDMS is known to interact with hydrophobic chemicals. This interaction is exacerbated at the scale of microfluidics, making careful modeling of in-device concentrations vital for PDMS-based microfluidic devices. While it has been previously reported that many chemicals diffuse through PDMS, here we report that diffusion in PDMS is anomalous, i.e. characterized by…

Patterns Within the Markov Tree
Robert A. Gore
https://arxiv.org/abs/2506.04299 https://arxiv.org/pdf/2506.04299

Patterns Within the Markov Tree
An analysis of the Markov tree is presented. Markov triplets, {x,R,z}, are the positive integer solutions to the Diophantine equation x2 + R2 + z2 = 3xRz. Inspired by patterns of the Fibonacci and Pell triplets in Region 1 and Region 2 of the tree, an investigation of interior regions of the Markov tree finds generating functions and sequence functions for all triplets of all regions. These sequence functions lead to the discovery of a Pell equation for the Markov region numbers along the edges…

A Unified Theory of Compositionality, Modularity, and Interpretability in Markov Decision Processes
Thomas J. Ringstrom, Paul R. Schrater
https://arxiv.org/abs/2506.09499

A Unified Theory of Compositionality, Modularity, and Interpretability in Markov Decision Processes
We introduce Option Kernel Bellman Equations (OKBEs) for a new reward-free Markov Decision Process. Rather than a value function, OKBEs directly construct and optimize a predictive map called a state-time option kernel (STOK) to maximize the probability of completing a goal while avoiding constraint violations. STOKs are compositional, modular, and interpretable initiation-to-termination transition kernels for policies in the Options Framework of Reinforcement Learning. This means: 1) STOKs can…

Replaced article(s) found for q-bio.PE. https://arxiv.org/list/q-bio.PE/new
[1/1]:
- Hidden Markov modelling of spatio-temporal dynamics of measles in 1750-1850 Finland
Tiia-Maria Pasanen, Jouni Helske, Tarmo Ketola

Level-3 large deviations for the white-forced 2D Navier-Stokes system in a bounded domain
Meng Zhao
https://arxiv.org/abs/2506.14119 https://

Level-3 large deviations for the white-forced 2D Navier-Stokes system in a bounded domain
We study the large deviations principle (LDP) of Donsker-Varadhan type for the white-forced Navier-Stokes system in a bounded domain. Under the assumption that the noise is non-degenerate, we establish level-2 and level-3 LDPs with rate functions given by the Donsker-Varadhan formulas. The proof relies on an improved version of Kifer's criterion, a lift argument inspired from [DV83], an improved abstract result on the large-time asymptotics of generalized Markov semigroups, and a delicate appro…

Ergodic Theory of Inhomogeneous Quantum Processes
Abdessatar Souissi
https://arxiv.org/abs/2506.12280 https://arxiv.org/pdf/2506.1228…

Ergodic Theory of Inhomogeneous Quantum Processes
This work presents a rigorous framework for understanding ergodicity and mixing in time-inhomogeneous quantum systems. By analyzing quantum processes driven by sequences of quantum channels, we distinguish between forward and backward dynamics and reveal their fundamental asymmetries. A key contribution is the development of a quantum Markov-Dobrushin approach to characterize mixing behavior, offering clear criteria for convergence and exponential stability. The framework not only generalizes c…

Efficient Remote Monitoring through Noisy Random Access with Retransmissions
Sergey Foss, Dmitriy Kim, Andrey Turlikov
https://arxiv.org/abs/2507.12368 htt…

Efficient Remote Monitoring through Noisy Random Access with Retransmissions
We consider a rare event monitoring system consisting of a set of devices and a base station, where devices transmit information about rare events to the base station using a random multiple access scheme. We introduce a model in which the presence of noise in the multiple access channel can cause message loss even in the absence of transmission collisions. The occurrence of events is modeled by a family of independent two-state Markov chains (with states 0 and 1). We analyze how repeated trans…

On characterization and existence of a constrained correlated equilibria in Markov games
Tingting Ni, Anna Maddux, Maryam Kamgarpour
https://arxiv.org/abs/2507.03502

On characterization and existence of a constrained correlated equilibria in Markov games
Markov games with coupling constraints provide a natural framework to study constrained decision-making involving self-interested agents, where the feasibility of an individual agent's strategy depends on the joint strategies of the others. Such games arise in numerous real-world applications involving safety requirements and budget caps, for example, in environmental management, electricity markets, and transportation systems. While correlated equilibria have emerged as an important solution c…

Markov Blanket Density and Free Energy Minimization
Luca M. Possati
https://arxiv.org/abs/2506.05794 https://arxiv.org/pdf/2506.05794…

Markov Blanket Density and Free Energy Minimization
This paper presents a continuous, information-theoretic extension of the Free Energy Principle through the concept of Markov blanket density, i.e., a scalar field that quantifies the degree of conditional independence between internal and external states at each point in space (ranging from 0 for full coupling to 1 for full separation). It demonstrates that active inference dynamics (including the minimization of variational and expected free energy) naturally emerge from spatial gradients in t…

Markov Regime-Switching Intelligent Driver Model for Interpretable Car-Following Behavior
Chengyuan Zhang, Cathy Wu, Lijun Sun
https://arxiv.org/abs/2506.14762

Markov Regime-Switching Intelligent Driver Model for Interpretable Car-Following Behavior
Accurate and interpretable car-following models are essential for traffic simulation and autonomous vehicle development. However, classical models like the Intelligent Driver Model (IDM) are fundamentally limited by their parsimonious and single-regime structure. They fail to capture the multi-modal nature of human driving, where a single driving state (e.g., speed, relative speed, and gap) can elicit many different driver actions. This forces the model to average across distinct behaviors, red…

Large deviations for general Markov chains on discrete state spaces
L\'eo Daures
https://arxiv.org/abs/2507.11166 https://arxiv.o…

Large deviations for general Markov chains on discrete state spaces
We study the large deviations of Markov chains under the sole assumption that the state space is discrete. In particular, we do not require any of the usual irreducibility and exponential tightness assumptions. Using subadditive arguments, we provide an elementary and self-contained proof of the level-2 and level-3 large deviation principles. Due to the possible reducibility of the Markov chain, the rate functions may be nonconvex and may differ, outside a specific set, from the Donsker-Varadha…

The Gauss-Markov Adjunction: Categorical Semantics of Residuals in Supervised Learning
Moto Kamiura
https://arxiv.org/abs/2507.02442 https://

The Gauss-Markov Adjunction: Categorical Semantics of Residuals in Supervised Learning
Enhancing the intelligibility and interpretability of machine learning is a crucial task in responding to the demand for Explicability as an AI principle, and in promoting the better social implementation of AI. The aim of our research is to contribute to this improvement by reformulating machine learning models through the lens of category theory, thereby developing a semantic framework for structuring and understanding AI systems. Our categorical modeling in this paper clarifies and formalize…

Replaced article(s) found for math.CV. https://arxiv.org/list/math.CV/new
[1/1]:
Holomorphic automorphisms of Markov-type surfaces
https://

Recent Advances in Simulation-based Inference for Gravitational Wave Data Analysis
Bo Liang, He Wang
https://arxiv.org/abs/2507.11192 https://

Recent Advances in Simulation-based Inference for Gravitational Wave Data Analysis
The detection of gravitational waves by the LIGO-Virgo-KAGRA collaboration has ushered in a new era of observational astronomy, emphasizing the need for rapid and detailed parameter estimation and population-level analyses. Traditional Bayesian inference methods, particularly Markov chain Monte Carlo, face significant computational challenges when dealing with the high-dimensional parameter spaces and complex noise characteristics inherent in gravitational wave data. This review examines the em…

Machine Learning Acceleration of Neutron Star Pulse Profile Modeling
Preston G. Waldrop, Dimitrios Psaltis, Tong Zhao
https://arxiv.org/abs/2506.11194 http…

Machine Learning Acceleration of Neutron Star Pulse Profile Modeling
Ray tracing algorithms that compute pulse profiles from rotating neutron stars are essential tools for constraining neutron-star properties with data from missions such as NICER. However, the high computational cost of these simulations presents a significant bottleneck for inference algorithms that require millions of evaluations, such as Markov Chain Monte Carlo methods. In this work, we develop a residual neural network model that accelerates this calculation by predicting the observed flux …

Bonded braids and the Markov theorem
Paolo Cavicchioli, Bo\v{s}tjan Gabrov\v{s}ek, Matic Simoni\v{c}
https://arxiv.org/abs/2507.04565 https://

Bonded braids and the Markov theorem
Bonded knots arise naturally in topological protein modeling, where intramolecular interactions such as disulfide bridges stabilize folded configurations. These structures extend classical knot theory by incorporating embedded graphs, and have been formalized as bonded knots. In this paper, we develop the algebraic theory of bonded braids, introducing the bonded braid monoid in the topological and rigid settings, which encodes both classical braid crossings and (rigid) bonded connections. We pr…

Ratio limits and pressure function for group extensions of Gibbs Markov maps
Jaime Gomez, Dalia Terhesiu
https://arxiv.org/abs/2507.08186 https://

Ratio limits and pressure function for group extensions of Gibbs Markov maps
Ratio limit theorems for random walks on (various) groups are known. We obtain a generalization of this type of ratio limit for deterministic walks on certain groups driven by Gibbs Markov maps. In terms of proofs, the main difficulty comes down to the absence of a convolution structure. Also, for (finitely generated) group extensions of Gibbs Markov maps we obtain a characterization of the pressure function without a symmetry assumption.

Feature-Based Belief Aggregation for Partially Observable Markov Decision Problems
Yuchao Li, Kim Hammar, Dimitri Bertsekas
https://arxiv.org/abs/2507.04646

Feature-Based Belief Aggregation for Partially Observable Markov Decision Problems
We consider a finite-state partially observable Markov decision problem (POMDP) with an infinite horizon and a discounted cost, and we propose a new method for computing a cost function approximation that is based on features and aggregation. In particular, using the classical belief-space formulation, we construct a related Markov decision problem (MDP) by first aggregating the unobservable states into feature states, and then introducing representative beliefs over these feature states. This …

Post-processing of EEG-based Auditory Attention Decoding Decisions via Hidden Markov Models
Nicolas Heintz, Tom Francart, Alexander Bertrand
https://arxiv.org/abs/2506.24024

Post-processing of EEG-based Auditory Attention Decoding Decisions via Hidden Markov Models
Auditory attention decoding (AAD) algorithms exploit brain signals, such as electroencephalography (EEG), to identify which speaker a listener is focusing on in a multi-speaker environment. While state-of-the-art AAD algorithms can identify the attended speaker on short time windows, their predictions are often too inaccurate for practical use. In this work, we propose augmenting AAD with a hidden Markov model (HMM) that models the temporal structure of attention. More specifically, the HMM rel…

Supersymmetric properties of one-dimensional Markov generators with the links to Markov-dualities and to shape-invariance-exact-solvability
Cecile Monthus
https://arxiv.org/abs/2507.04941

Supersymmetric properties of one-dimensional Markov generators with the links to Markov-dualities and to shape-invariance-exact-solvability
For diffusion process involving the force $F(x)$ and the diffusion coefficient $D(x)$, the continuity equation $\partial_t P_t(x)=- \partial_xJ_t(x)$ gives the dynamics of the probability $P_t( x)$ in terms of the current $J_t( x)=F(x)P_t(x)-D(x)\partial_x P_t(x)={\cal J}P_t( x)$ obtained from $P_t( x) $ via the application of the first-order differential current-operator ${\cal J}$. So the dynamics of the probability $P_t( x)$ is governed by the factorized Fokker-Planck generator ${\cal F}=-\p…

Parallel computations for Metropolis Markov chains with Picard maps
Sebastiano Grazzi, Giacomo Zanella
https://arxiv.org/abs/2506.09762 https://

Parallel computations for Metropolis Markov chains with Picard maps
We develop parallel algorithms for simulating zeroth-order (aka gradient-free) Metropolis Markov chains based on the Picard map. For Random Walk Metropolis Markov chains targeting log-concave distributions $π$ on $\mathbb{R}^d$, our algorithm generates samples close to $π$ in $\mathcal{O}(\sqrt{d})$ parallel iterations with $\mathcal{O}(\sqrt{d})$ processors, therefore speeding up the convergence of the corresponding sequential implementation by a factor $\sqrt{d}$. Furthermore, a modificatio…

Markov Perfect Equilibria in Discrete Finite-Player and Mean-Field Games
Felix H\"ofer, H. Mete Soner, Atilla Y{\i}lmaz
https://arxiv.org/abs/2507.04540

Markov Perfect Equilibria in Discrete Finite-Player and Mean-Field Games
We study dynamic finite-player and mean-field stochastic games within the framework of Markov perfect equilibria (MPE). Our focus is on discrete time and space structures without monotonicity. Unlike their continuous-time analogues, discrete-time finite-player games generally do not admit unique MPE. However, we show that uniqueness is remarkably recovered when the time steps are sufficiently small, and we provide examples demonstrating the necessity of this assumption. This result, established…

This https://arxiv.org/abs/2504.06592 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLO_…

On Coalgebraic Product Constructions for Markov Chains and Automata
Verifying traces of systems is a central topic in formal verification. We study model checking of Markov chains (MCs) against temporal properties represented as (finite) automata. For instance, given an MC and a deterministic finite automaton (DFA), a simple but practically useful model checking problem asks for the probability of traces on the MC that are accepted by the DFA. A standard approach to solving this problem constructs a product MC of the given MC and DFA, reducing the task to a sim…

Drawdowns, Drawups, and Occupation Times under General Markov Models
Pingping Zeng, Gongqiu Zhang, Weinan Zhang
https://arxiv.org/abs/2506.00552 https://…

Drawdowns, Drawups, and Occupation Times under General Markov Models
Drawdown risk, an important metric in financial risk management, poses significant computational challenges due to its highly path-dependent nature. This paper proposes a unified framework for computing five important drawdown quantities introduced in Landriault et al. (2015) and Zhang (2015) under general Markov models. We first establish linear systems and develop efficient algorithms for such problems under continuous-time Markov chains (CTMCs), and then establish their theoretical convergen…

Transformers Beyond Order: A Chaos-Markov-Gaussian Framework for Short-Term Sentiment Forecasting of Any Financial OHLC timeseries Data
Arif Pathan
https://arxiv.org/abs/2506.17244

Transformers Beyond Order: A Chaos-Markov-Gaussian Framework for Short-Term Sentiment Forecasting of Any Financial OHLC timeseries Data
Short-term sentiment forecasting in financial markets (e.g., stocks, indices) is challenging due to volatility, non-linearity, and noise in OHLC (Open, High, Low, Close) data. This paper introduces a novel CMG (Chaos-Markov-Gaussian) framework that integrates chaos theory, Markov property, and Gaussian processes to improve prediction accuracy. Chaos theory captures nonlinear dynamics; the Markov chain models regime shifts; Gaussian processes add probabilistic reasoning. We enhance the framework…

Affine invariant interacting Langevin dynamics in Markov chain importance sampling for rare event estimation
Jason Beh, J\'er\^ome Morio, Florian Simatos, Simon Weissmann
https://arxiv.org/abs/2506.20185

Affine invariant interacting Langevin dynamics in Markov chain importance sampling for rare event estimation
This work considers the framework of Markov chain importance sampling~(MCIS), in which one employs a Markov chain Monte Carlo~(MCMC) scheme to sample particles approaching the optimal distribution for importance sampling, prior to estimating the quantity of interest through importance sampling. In rare event estimation, the optimal distribution admits a non-differentiable log-density, thus gradient-based MCMC can only target a smooth approximation of the optimal density. We propose a new gradie…

Improving Reinforcement Learning Sample-Efficiency using Local Approximation
Mohit Prashant, Arvind Easwaran
https://arxiv.org/abs/2507.12383 https://

Improving Reinforcement Learning Sample-Efficiency using Local Approximation
In this study, we derive Probably Approximately Correct (PAC) bounds on the asymptotic sample-complexity for RL within the infinite-horizon Markov Decision Process (MDP) setting that are sharper than those in existing literature. The premise of our study is twofold: firstly, the further two states are from each other, transition-wise, the less relevant the value of the first state is when learning the $ε$-optimal value of the second; secondly, the amount of 'effort', sample-complexity-wise, ex…

Preliminary analysis of Sus scrofa movement using Hidden Markov Models and Networks
Riccardo Basilone, Eleonora Bergamin, Federica Fanelli, Egor Kotov, Kevin Morelle, Alisa Klamm, Manal Nhili, Joshua Rosen, Andrew Schendl, Olena Holubowska, Andrew Renninger, Kamil Smolak
https://arxiv.org/abs/2506.22138

Preliminary analysis of Sus scrofa movement using Hidden Markov Models and Networks
This study examines the complex movement patterns and behavioral characteristics of wild boars using GPS telemetry data collected over a two-month period. Our methodological approach centers on the application of a Hidden Markov Model (HMM) to discern distinct behavioral states embedded within the trajectories. Furthermore, the study aimed to construct behavioral networks, derived from these segmented trajectories. The resultant network structures showed that the hidden behavioral patterns are …

@… Have you considered asking a tiny Markov chain, trained on the works of arseholes on 4chan, to respond for you?

This https://arxiv.org/abs/2403.15425 has been replaced.
initial toot: https://mastoxiv.page/@ar…

Multistep reversible excitation transfer in a multicomponent rigid solution: II. Modeling the dynamics of radiationless transfer as a time-resolved Markov chain
To determine the effect of nonradiative excitation energy transfer on the fluorescence of a rigid multicomponent solution, a new analytical method was developed by treating this transfer as a time-resolved Markov chain (TRMC). In the TRMC approach, we assume that the Markov chain under consideration is governed by bivariate joint probability mass-density functions. One of the random variables is discrete and represents the state number to which the process passes at a given step, while the othe…

Speeding Up Hyper-Heuristics With Markov-Chain Operator Selection and the Only-Worsening Acceptance Operator
Abderrahim Bendahi, Benjamin Doerr, Adrien Fradin, Johannes F. Lutzeyer
https://arxiv.org/abs/2506.01107

Speeding Up Hyper-Heuristics With Markov-Chain Operator Selection and the Only-Worsening Acceptance Operator
The move-acceptance hyper-heuristic was recently shown to be able to leave local optima with astonishing efficiency (Lissovoi et al., Artificial Intelligence (2023)). In this work, we propose two modifications to this algorithm that demonstrate impressive performances on a large class of benchmarks including the classic Cliff$_d$ and Jump$_m$ function classes. (i) Instead of randomly choosing between the only-improving and any-move acceptance operator, we take this choice via a simple two-state…

Probabilistic Modeling of Antibody Kinetics Post Infection and Vaccination: A Markov Chain Approach
Rayanne A. Luke, Prajakta Bedekar, Lyndsey M. Muehling, Glenda Canderan, Yesun Lee, Wesley A. Cheng, Judith A. Woodfolk, Jeffrey M. Wilson, Pia S. Pannaraj, Anthony J. Kearsley
https://arxiv.org/abs/2507.10793

Probabilistic Modeling of Antibody Kinetics Post Infection and Vaccination: A Markov Chain Approach
Understanding the dynamics of antibody levels is crucial for characterizing the time-dependent response to immune events: either infections or vaccinations. The sequence and timing of these events significantly influence antibody level changes. Despite extensive interest in the topic in the recent years and many experimental studies, the effect of immune event sequences on antibody levels is not well understood. Moreover, disease or vaccination prevalence in the population are time-dependent. T…

Lower Bounds on the Size of Markov Equivalence Classes
Erik Jahn, Frederick Eberhardt, Leonard J. Schulman
https://arxiv.org/abs/2506.20933 https://…

Lower Bounds on the Size of Markov Equivalence Classes
Causal discovery algorithms typically recover causal graphs only up to their Markov equivalence classes unless additional parametric assumptions are made. The sizes of these equivalence classes reflect the limits of what can be learned about the underlying causal graph from purely observational data. Under the assumptions of acyclicity, causal sufficiency, and a uniform model prior, Markov equivalence classes are known to be small on average. In this paper, we show that this is no longer the ca…

The effects of initial conditions on the accuracy of mean-field approximations of Markov processes on large random graphs
Pierfrancesco Dionigi, D\'aniel Keliger
https://arxiv.org/abs/2506.12872

The effects of initial conditions on the accuracy of mean-field approximations of Markov processes on large random graphs
We study the evolution of a general class of stochastic processes (containing, e.g. SIS and SIR models) on large random networks, focusing on a particular general class of random graph models. To approximate the expected dynamics of these processes, we employ a mean-field technique known as the N-Intertwined Mean-Field Approximation (NIMFA). Our primary goal is to quantify the impact of the randomness in the network topology on the accuracy of such approximations. While classical work by Kurtz …

Probability-Raising Causality for Uncertain Parametric Markov Decision Processes with PAC Guarantees
Ryohei Oura, yuji Ito
https://arxiv.org/abs/2507.07319

Probability-Raising Causality for Uncertain Parametric Markov Decision Processes with PAC Guarantees
Recent decision-making systems are increasingly complicated, making it crucial to verify and understand their behavior for a given specification. A promising approach is to comprehensively explain undesired behavior in the systems modeled by Markov decision processes (MDPs) through formal verification and causal reasoning. However, the reliable explanation using model-based probabilistic causal analysis has not been explored when the MDP's transition probabilities are uncertain. This paper prop…

Thermodynamic control of non-equilibrium systems
Dana Kamp, Karel Proesmans
https://arxiv.org/abs/2506.14416 https://arxiv.org/pdf/25…

Thermodynamic control of non-equilibrium systems
We study the thermodynamic cost associated with driving systems between different non-equilibrium steady states. In particular, we combine a linear-response framework for non-equilibrium Markov systems with Lagrangian techniques to minimize the dissipation associated with driving processes. We then apply our framework to a simple toy model. Our results show several remarkable properties for the optimal protocol, such as diverging parameters and finite entropy production in the slow-driving limi…

Scalable Variational Inference for Multinomial Probit Models under Large Choice Sets and Sample Sizes
Gyeongjun Kim, Yeseul Kang, Lucas Kock, Prateek Bansal, Keemin Sohn
https://arxiv.org/abs/2507.10945

Scalable Variational Inference for Multinomial Probit Models under Large Choice Sets and Sample Sizes
The multinomial probit (MNP) model is widely used to analyze categorical outcomes due to its ability to capture flexible substitution patterns among alternatives. Conventional likelihood based and Markov chain Monte Carlo (MCMC) estimators become computationally prohibitive in high dimensional choice settings. This study introduces a fast and accurate conditional variational inference (CVI) approach to calibrate MNP model parameters, which is scalable to large samples and large choice sets. A f…

Solving Zero-Sum Convex Markov Games
Fivos Kalogiannis, Emmanouil-Vasileios Vlatakis-Gkaragkounis, Ian Gemp, Georgios Piliouras
https://arxiv.org/abs/2506.16120

Solving Zero-Sum Convex Markov Games
We contribute the first provable guarantees of global convergence to Nash equilibria (NE) in two-player zero-sum convex Markov games (cMGs) by using independent policy gradient methods. Convex Markov games, recently defined by Gemp et al. (2024), extend Markov decision processes to multi-agent settings with preferences that are convex over occupancy measures, offering a broad framework for modeling generic strategic interactions. However, even the fundamental min-max case of cMGs presents signi…

EnerBridge-DPO: Energy-Guided Protein Inverse Folding with Markov Bridges and Direct Preference Optimization
Dingyi Rong, Haotian Lu, Wenzhuo Zheng, Fan Zhang, Shuangjia Zheng, Ning Liu
https://arxiv.org/abs/2506.09496

EnerBridge-DPO: Energy-Guided Protein Inverse Folding with Markov Bridges and Direct Preference Optimization
Designing protein sequences with optimal energetic stability is a key challenge in protein inverse folding, as current deep learning methods are primarily trained by maximizing sequence recovery rates, often neglecting the energy of the generated sequences. This work aims to overcome this limitation by developing a model that directly generates low-energy, stable protein sequences. We propose EnerBridge-DPO, a novel inverse folding framework focused on generating low-energy, high-stability prot…

This https://arxiv.org/abs/2412.00193 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_qu…

Spacetime Markov length: a diagnostic for fault tolerance via mixed-state phases
We establish a correspondence between the fault-tolerance of local stabilizer codes experiencing measurement and physical errors and the mixed-state phases of decohered resource states in one higher dimension. Drawing from recent developments in mixed-state phases of matter, this motivates a diagnostic of fault-tolerance, which we refer to as the spacetime Markov length. This is a length scale determined by the decay of the (classical) conditional mutual information of repeated syndrome measure…

An elementary proof of existence and uniqueness of stationary distributions for irreducible Markov chains
Rinaldo B. Schinazi
https://arxiv.org/abs/2506.13662

An elementary proof of existence and uniqueness of stationary distributions for irreducible Markov chains
Consider an $n\times n$ matrix $P$ with the following properties. All entries in $P$ are positive or $0$, the sum of each row is 1 and for all $i$ and $j$ in $\{1,\dots,n\}$ there exists a natural number $k$ such that the $(i,j)$ entry of the matrix $P^k$ is strictly positive. Then, there exists a unique row vector $v$ with only strictly positive entries, whose sum of entries is 1 and such that $vP=P$. We present a proof of this well-known result that uses only basic algebra and the Bolzano-Wei…

Accelerating seismic inversion and uncertainty quantification with efficient high-rank Hessian approximations
Mathew Hu, Nick Alger, Rami Nammour, Omar Ghattas
https://arxiv.org/abs/2507.10804

Accelerating seismic inversion and uncertainty quantification with efficient high-rank Hessian approximations
Efficient high-rank approximations of the Hessian can accelerate seismic full waveform inversion (FWI) and uncertainty quantification (UQ). In FWI, approximations of the inverse of the Hessian may be used as preconditioners for Newton-type or quasi-Newton algorithms, reducing computational costs and improving recovery in deeper subsurface regions. In Bayesian UQ, Hessian approximations enable the construction of Markov chain Monte Carlo (MCMC) proposals that capture the directional scalings of …

Extendable Shift Maps and Weighted Endomorphisms on Generalized Countable Markov Shifts
Rodrigo Bissacot, Iv\'an Diaz-Granados, Thiago Raszeja
https://arxiv.org/abs/2506.07487

Extendable Shift Maps and Weighted Endomorphisms on Generalized Countable Markov Shifts
We obtain an operator algebraic characterization for when we can continuously extend the shift map from a standard countable Markov shift $Σ_A$ to its respective generalized countable Markov shift $X_A$ (a compactification of $Σ_A$). When the shift map is continuously extendable, we obtain explicit formulas for the spectral radius of weighted endomorphisms $aα$, where $α$ is dual to the shift map and conjugated to $Θ(f)=f \circ σ$ on $C(X_A)$, extending a theorem of Kwaśniewski and Lebed…

On the validity of the Radon-Nikodym Theorem
Paolo Roselli, Michel Willem
https://arxiv.org/abs/2506.12023 https://arxiv.org/pdf/2506…

On the validity of the Radon-Nikodym Theorem
This paper presents a new general formulation of the Radon-Nikodym theorem in the setting of abstract measure theory. We introduce the notion of weak localizability for a measure and show that this property is both necessary and sufficient for the validity of a Radon-Nikodym-type representation under a natural compatibility relation between measures. The proof relies solely on elementary tools, such as Markov's inequality and the monotone convergence theorem. In addition to establishing the mai…

Bernstein-Markov measures and Toeplitz theory
Siarhei Finski
https://arxiv.org/abs/2506.01610 https://arxiv.org/pdf/2506.01610…

Bernstein-Markov measures and Toeplitz theory
We prove that Toeplitz operators associated with a Bernstein-Markov measure on a compact complex manifold endowed with a big line bundle form an algebra under composition. As an application, we derive a Szegő-type spectral equidistribution result for this class of operators. A key component of our approach is the off-diagonal asymptotic analysis of the Bergman kernel, also known as the Christoffel-Darboux kernel.

An infinite horizon sufficient stochastic maximum principle for regime switching diffusions and applications
Kai Ding, Xun Li, Siyu Lv, Xin Zhang
https://arxiv.org/abs/2506.11523 …

An infinite horizon sufficient stochastic maximum principle for regime switching diffusions and applications
This paper is concerned with a discounted stochastic optimal control problem for regime switching diffusion in an infinite horizon. First, as a preliminary with particular interests in its own right, the global well-posedness of infinite horizon forward and backward stochastic differential equations with Markov chains and the asymptotic property of their solutions when time goes to infinity are obtained. Then, a sufficient stochastic maximum principle for optimal controls is established via a d…

Expressivity of bisimulation pseudometrics over analytic state spaces
Daniel Luckhardt, Harsh Beohar, Clemens Kupke
https://arxiv.org/abs/2505.23635 https:…

Expressivity of bisimulation pseudometrics over analytic state spaces
A Markov decision process (MDP) is a state-based dynamical system capable of describing probabilistic behaviour with rewards. In this paper, we view MDPs as coalgebras living in the category of analytic spaces, a very general class of measurable spaces. Note that analytic spaces were already studied in the literature on labelled Markov processes and bisimulation relations. Our results are twofold. First, we define bisimulation pseudometrics over such coalgebras using the framework of fibrations…

This https://arxiv.org/abs/2504.01759 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Alpha-Beta HMM: Hidden Markov Model Filtering with Equal Exit Probabilities and a Step-Size Parameter
The hidden Markov model (HMM) provides a powerful framework for inference in time-varying environments, where the underlying state evolves according to a Markov chain. To address the optimal filtering problem in general dynamic settings, we propose the $αβ$-HMM algorithm, which simplifies the state transition model to a Markov chain with equal exit probabilities and introduces a step-size parameter to balance the influence of observational data and the model. By analyzing the algorithm's dyna…

Reversible Imprecise Markov Chains
Damjan \v{S}kulj
https://arxiv.org/abs/2507.06374 https://arxiv.org/pdf/2507.06374

Reversible Imprecise Markov Chains
Reversible Markov chains have identical forward and backward path distributions; this symmetry yields tractable stationary behaviour and powers algorithms from MCMC to network random-walk analysis. We extend reversibility to imprecise Markov chains, in which marginal and transition probabilities are specified by credal sets. Adopting the strong-independence interpretation of the Markov property, we reverse every precise chain compatible with a given credal specification and study the resulting …

A model-agnostic gravitational-wave background characterization algorithm
Taylor Knapp, Patrick M. Meyers, Arianna I. Renzini
https://arxiv.org/abs/2507.08095

A model-agnostic gravitational-wave background characterization algorithm
As ground-based gravitational-wave (GW) detectors improve in sensitivity, gravitational-wave background (GWB) signals will progressively become detectable. Currently, searches for the GWB model the signal as a power law, however deviations from this model will be relevant at increased sensitivity. Therefore, to prepare for the range of potentially detectable GWB signals, we propose an interpolation model implemented through a transdimensional reverse-jump Markov chain Monte Carlo (RJMCMC) algor…

Stochastic thermodynamics for classical non-Markov jump processes
Kiyoshi Kanazawa, Andreas Dechant
https://arxiv.org/abs/2506.04726 https://

Stochastic thermodynamics for classical non-Markov jump processes
Stochastic thermodynamics investigates energetic/entropic bounds in small systems, such as biomolecular motors, chemical-reaction networks, and quantum nano-devices. Foundational results, including the second law and thermodynamic uncertainty relations, predominantly rely on the Markov assumption -- neglecting history dependence of physical systems. However, while physicists recognise that the Markov assumption is dubious in real experimental setups, extending stochastic thermodynamics to gener…

Dealing with separation problem in hidden Markov models with covariates based on a penalized maximum likelihood approach
Luca Brusa, Fulvia Pennoni, Francesco Bartolucci, Romina Peruilh Bagolini
https://arxiv.org/abs/2507.02425

Dealing with separation problem in hidden Markov models with covariates based on a penalized maximum likelihood approach
A penalized maximum likelihood estimation approach is proposed for discrete-time hidden Markov models where covariates affect the observed responses and serial dependence is considered. The proposed penalized maximum likelihood method addresses the issue of latent state separation that typically occurs when this model is applied to binary and categorical response variables with a limited number of categories, resulting in extremely large estimates of the support points of the latent variable as…

Searching for a Hidden Markov Anomaly over Multiple Processes
Levli Citron, Kobi Cohen, Qing Zhao
https://arxiv.org/abs/2506.17108 https://

Searching for a Hidden Markov Anomaly over Multiple Processes
We address the problem of detecting an anomalous process among a large number of processes. At each time t, normal processes are in state zero (normal state), while the abnormal process may be in either state zero (normal state) or state one (abnormal state), with the states being hidden. The transition between states for the abnormal process is governed by a Markov chain over time. At each time step, observations can be drawn from a selected subset of processes. Each probed process generates a…

Homogenization of Multi-agent Learning Dynamics in Finite-state Markov Games
Yann Kerzreho (ENS Paris Saclay)
https://arxiv.org/abs/2506.21079 https://

Homogenization of Multi-agent Learning Dynamics in Finite-state Markov Games
This paper introduces a new approach for approximating the learning dynamics of multiple reinforcement learning (RL) agents interacting in a finite-state Markov game. The idea is to rescale the learning process by simultaneously reducing the learning rate and increasing the update frequency, effectively treating the agent's parameters as a slow-evolving variable influenced by the fast-mixing game state. Under mild assumptions-ergodicity of the state process and continuity of the updates-we prov…

Analysis and Optimization of Probabilities of Beneficial Mutation and Crossover Recombination in a Hamming Space
Roman V. Belavkin
https://arxiv.org/abs/2506.13809

Analysis and Optimization of Probabilities of Beneficial Mutation and Crossover Recombination in a Hamming Space
Inspired by Fisher's geometric approach to study beneficial mutations, we analyse probabilities of beneficial mutation and crossover recombination of strings in a general Hamming space with arbitrary finite alphabet. Mutations and recombinations that reduce the distance to an optimum are considered as beneficial. Geometric and combinatorial analysis is used to derive closed-form expressions for transition probabilities between spheres around an optimum giving a complete description of Markov ev…

An approximation theory for Markov chain compression
Mark Fornace, Michael Lindsey
https://arxiv.org/abs/2506.22918 https://arxiv.org…

An approximation theory for Markov chain compression
We develop a framework for the compression of reversible Markov chains with rigorous error control. Given a subset of selected states, we construct reduced dynamics that can be lifted to an approximation of the full dynamics, and we prove simple spectral and nuclear norm bounds on the recovery error in terms of a suitably interpreted Nyström approximation error. We introduce two compression schemes: a projective compression based on committor functions and a structure-preserving compression de…

A universal constraint for relaxation rates for quantum Markov generators: complete positivity and beyond
Dariusz Chru\'sci\'nski, Frederik vom Ende, Gen Kimura, Paolo Muratore-Ginanneschi
https://arxiv.org/abs/2505.24467

A universal constraint for relaxation rates for quantum Markov generators: complete positivity and beyond
Relaxation rates are key characteristics of quantum processes, as they determine how quickly a quantum system thermalizes, equilibrates, decoheres, and dissipates. While they play a crucial role in theoretical analyses, relaxation rates are also often directly accessible through experimental measurements. Recently, it was shown that for quantum processes governed by Markovian semigroups, the relaxation rates satisfy a universal constraint: the maximal rate is upper-bounded by the sum of all rat…

On Limiting Probability Distributions of Higher Order Markov Chains
Lixing Han, Jianhong Xu
https://arxiv.org/abs/2506.08874 https://…

On Limiting Probability Distributions of Higher Order Markov Chains
The limiting (or stationary) probability distribution is one of the key characteristics of a Markov chain since it shows its long-term behavior. In this paper, for a higher order Markov chain, we establish a sufficient condition for the existence of its limiting probability distribution. This condition is built upon the regularity of its transition tensor. Our results extend the corresponding conclusions for first order chains. Besides, they complement the existing results concerning higher ord…

Convex Approximations of Random Constrained Markov Decision Processes
V Varagapriya, Vikas Vikram Singh, Abdel Lisser
https://arxiv.org/abs/2505.24815 http…

Convex Approximations of Random Constrained Markov Decision Processes
Constrained Markov decision processes (CMDPs) are used as a decision-making framework to study the long-run performance of a stochastic system. It is well-known that a stationary optimal policy of a CMDP problem under discounted cost criterion can be obtained by solving a linear programming problem when running costs and transition probabilities are exactly known. In this paper, we consider a discounted cost CMDP problem where the running costs and transition probabilities are defined using ran…

Tensor-product interactions in Markov-switching models
Jan-Ole Koslik
https://arxiv.org/abs/2507.01555 https://arxiv.org/pdf/2507.015…

Tensor-product interactions in Markov-switching models
Markov-switching models are a powerful tool for modelling time series data that are driven by underlying latent states. As such, they are widely used in behavioural ecology, where discrete states can serve as proxies for behavioural modes and enable inference on latent behaviour driving e.g. observed movement. To understand drivers of behavioural changes, it is common to link model parameters to covariates. Over the last decade, nonparametric approaches have gained traction in this context to a…

This https://arxiv.org/abs/2503.21224 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

Efficient Learning for Entropy-Regularized Markov Decision Processes via Multilevel Monte Carlo
Designing efficient learning algorithms with complexity guarantees for Markov decision processes (MDPs) with large or continuous state and action spaces remains a fundamental challenge. We address this challenge for entropy-regularized MDPs with Polish state and action spaces, assuming access to a generative model of the environment. We propose a novel family of multilevel Monte Carlo (MLMC) algorithms that integrate fixed-point iteration with MLMC techniques and a generic stochastic approximat…

Omega-regular Verification and Control for Distributional Specifications in MDPs
S. Akshay (Dept of CSE, Indian Institute of Technology Bombay), Ouldouz Neysari (Singapore Management University, University of Tehran), {\DJ}or{\dj}e \v{Z}ikeli\'c (Singapore Management University)
https://arxiv.org/abs/2507.04286

Omega-regular Verification and Control for Distributional Specifications in MDPs
A classical approach to studying Markov decision processes (MDPs) is to view them as state transformers. However, MDPs can also be viewed as distribution transformers, where an MDP under a strategy generates a sequence of probability distributions over MDP states. This view arises in several applications, even as the probabilistic model checking problem becomes much harder compared to the classical state transformer counterpart. It is known that even distributional reachability and safety probl…

Markov Chains of Evolutionary Games with a Small Number of Players
Athanasios Kehagias
https://arxiv.org/abs/2506.23134 https://arxiv…

Markov Chains of Evolutionary Games with a Small Number of Players
We construct and study the transition probability matrix of evolutionary games in which the number of players is finite (and relatively small) of such games. We use a simplified version of the population games studied by Sandholm. After laying out a general framework we concentrate on specific examples, involving the Iterated Prisoner's Dilemma, the Iterated Stag Hunt, and the Rock-Paper-Scissors game. Also we consider several revision protocols: Best Response, Pairwise Comparison, Pairwise Pro…

On conditions for Dobrushin's Central limit theorem for non-homogeneous Markov chains
Alexander Veretennikov, Aisha Nurieva
https://arxiv.org/abs/2506.07287

On conditions for Dobrushin's Central limit theorem for non-homogeneous Markov chains
A new sufficient condition is proposed in the problem of Central Limit Theorem in the array scheme for non-homogeneous Markov Chains, with a possibility that the minimum of Markov-Dobrushin's ergodic coefficient may equal zero.

Wasserstein Distances on Quantum Structures: an Overview
Emily Beatty
https://arxiv.org/abs/2506.09794 https://arxiv.org/pdf/2506.097…

Wasserstein Distances on Quantum Structures: an Overview
The theory of optimal transport of probability measures has wide-ranging applications across a number of different fields, including concentration of measure, machine learning, Markov chains, and economics. The generalisation of optimal transport tools from probability measures to quantum states has shown great promise over the last few years, particularly in the development of the theory of Wasserstein-style distances and divergences between quantum states. Such distances have already led to a…

Recursive Privacy-Preserving Estimation Over Markov Fading Channels
Jie Huang, Fanlin Jia, Xiao He
https://arxiv.org/abs/2506.02725 https://

Recursive Privacy-Preserving Estimation Over Markov Fading Channels
In industrial applications, the presence of moving machinery, vehicles, and personnel, contributes to the dynamic nature of the wireless channel. This time variability induces channel fading, which can be effectively modeled using a Markov fading channel (MFC). In this paper, we investigate the problem of secure state estimation for systems that communicate over a MFC in the presence of an eavesdropper. The objective is to enable a remote authorized user to accurately estimate the states of a d…

On the subsystems 0f certain almost Markov shifts
Wolfgang Krieger
https://arxiv.org/abs/2507.02717 https://arxiv.org/pdf/2507.02717

On the subsystems 0f certain almost Markov shifts
For an aperiodic subshift of finite type Y and for a subshift X with topological entropy less than the topological entropy of Y, it was proved in Krieger: On the subsystems of topological Markov chains, Ergodic theory and dynamical systems 1982, 2, 195-202, that the necessary condition on the periodic points of X and Y for the existence of an embedding of X int Y is also sufficient for the existence of an embedding of X into Y. In this note we extend this theorem to a certain class of almost Ma…

This https://arxiv.org/abs/2411.08295 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Improving the convergence of Markov chains via permutations and projections
This paper aims at improving the convergence to equilibrium of finite ergodic Markov chains via permutations and projections. First, we prove that a specific mixture of permuted Markov chains arises naturally as a projection under the KL divergence or the squared-Frobenius norm. We then compare various mixing properties of the mixture with other competing Markov chain samplers and demonstrate that it enjoys improved convergence. This geometric perspective motivates us to propose samplers based …

Predictive Causal Inference via Spatio-Temporal Modeling and Penalized Empirical Likelihood
Byunghee Lee, Hye Yeon Sin, Joonsung Kang
https://arxiv.org/abs/2507.08896

Predictive Causal Inference via Spatio-Temporal Modeling and Penalized Empirical Likelihood
This study introduces an integrated framework for predictive causal inference designed to overcome limitations inherent in conventional single model approaches. Specifically, we combine a Hidden Markov Model (HMM) for spatial health state estimation with a Multi Task and Multi Graph Convolutional Network (MTGCN) for capturing temporal outcome trajectories. The framework asymmetrically treats temporal and spatial information regarding them as endogenous variables in the outcome regression, and e…

Transformers as Multi-task Learners: Decoupling Features in Hidden Markov Models
Yifan Hao, Chenlu Ye, Chi Han, Tong Zhang
https://arxiv.org/abs/2506.01919

Transformers as Multi-task Learners: Decoupling Features in Hidden Markov Models
Transformer based models have shown remarkable capabilities in sequence learning across a wide range of tasks, often performing well on specific task by leveraging input-output examples. Despite their empirical success, a comprehensive theoretical understanding of this phenomenon remains limited. In this work, we investigate the layerwise behavior of Transformers to uncover the mechanisms underlying their multi-task generalization ability. Taking explorations on a typical sequence model, i.e, H…

Gaussian Invariant Markov Chain Monte Carlo
Michalis K. Titsias, Angelos Alexopoulos, Siran Liu, Petros Dellaportas
https://arxiv.org/abs/2506.21511 https:…

Gaussian Invariant Markov Chain Monte Carlo
We develop sampling methods, which consist of Gaussian invariant versions of random walk Metropolis (RWM), Metropolis adjusted Langevin algorithm (MALA) and second order Hessian or Manifold MALA. Unlike standard RWM and MALA we show that Gaussian invariant sampling can lead to ergodic estimators with improved statistical efficiency. This is due to a remarkable property of Gaussian invariance that allows us to obtain exact analytical solutions to the Poisson equation for Gaussian targets. These …

Marginals of the planar symmetric Markov random flight on long time intervals behave like the Goldstein-Kac telegraph process
Alexander D. Kolesnik
https://arxiv.org/abs/2507.07525

Marginals of the planar symmetric Markov random flight on long time intervals behave like the Goldstein-Kac telegraph process
The planar symmetric Markov random flight $\bold X(t), \; t>0,$ is represented by the stochastic motion of a particle moving with constant finite speed $c>0$ in the Euclidean plane $\Bbb R^2$ and taking on its initial and each new directions at $λ$-Poisson ($λ>0$) distributed random time instants by choosing them at random according to the uniform distribution on the unit circumference. We consider the marginals of $\bold X(t)$, that is, the projection of this stochastic motion onto the axes.…

Learning The Minimum Action Distance
Lorenzo Steccanella, Joshua B. Evans, \"Ozg\"ur \c{S}im\c{s}ek, Anders Jonsson
https://arxiv.org/abs/2506.09276

Learning The Minimum Action Distance
This paper presents a state representation framework for Markov decision processes (MDPs) that can be learned solely from state trajectories, requiring neither reward signals nor the actions executed by the agent. We propose learning the minimum action distance (MAD), defined as the minimum number of actions required to transition between states, as a fundamental metric that captures the underlying structure of an environment. MAD naturally enables critical downstream tasks such as goal-conditi…

This https://arxiv.org/abs/2502.13697 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Linear programming for finite-horizon vector-valued Markov decision processes
We propose a vector linear programming formulation for a non-stationary, finite-horizon Markov decision process with vector-valued rewards. Pareto efficient policies are shown to correspond to efficient solutions of the linear program, and vector linear programming theory allows us to fully characterize deterministic efficient policies. An algorithm for enumerating all efficient deterministic policies is presented then tested numerically in an engineering application.

Some remarks on stochastic converse Lyapunov theorems
Pavel Osinenko, Grigory Yaremenko
https://arxiv.org/abs/2506.06053 https://arxi…

Some remarks on stochastic converse Lyapunov theorems
In this brief note, we investigate some constructions of Lyapunov functions for stochastic discrete-time stabilizable dynamical systems, in other words, controlled Markov chains. The main question here is whether a Lyapunov function in some statistical sense exists if the respective controlled Markov chain admits a stabilizing policy. We demonstrate some constructions extending on the classical results for deterministic systems. Some limitations of the constructed Lyapunov functions for stabili…

This https://arxiv.org/abs/2502.20799 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_qu…

Quantum-assisted variational Monte Carlo
Solving the ground state of quantum many-body systems remains a fundamental challenge in physics and chemistry. Recent advancements in quantum hardware have opened new avenues for addressing this challenge. Inspired by the quantum-enhanced Markov chain Monte Carlo (QeMCMC) algorithm [Nature, 619, 282-287 (2023)], which was originally designed for sampling the Boltzmann distribution of classical spin models using quantum computers, we introduce a quantum-assisted variational Monte Carlo (QA-VMC)…

The Law of Large Numbers and CLT for Non-stationary Markov Jump Processes Exhibiting Time-of-Day Effects
Monte Fischer, Peter W. Glynn
https://arxiv.org/abs/2506.08282

The Law of Large Numbers and CLT for Non-stationary Markov Jump Processes Exhibiting Time-of-Day Effects
In this paper, we develop a general law of large numbers and central limit theorem for finite state Markov jump processes with non-stationary transition rates. Such models commonly arise in service operations and manufacturing applications in which time-of-day, day-of-week, and secular effects are of first order importance in predicting system behavior. Our theorems allow for non-stationary reward environments that continuously accumulate reward, while including contributions from non-stationar…

Efficient Preference-Based Reinforcement Learning: Randomized Exploration Meets Experimental Design
Andreas Schlaginhaufen, Reda Ouhamma, Maryam Kamgarpour
https://arxiv.org/abs/2506.09508

Efficient Preference-Based Reinforcement Learning: Randomized Exploration Meets Experimental Design
We study reinforcement learning from human feedback in general Markov decision processes, where agents learn from trajectory-level preference comparisons. A central challenge in this setting is to design algorithms that select informative preference queries to identify the underlying reward while ensuring theoretical guarantees. We propose a meta-algorithm based on randomized exploration, which avoids the computational challenges associated with optimistic approaches and remains tractable. We e…

On Pioneering Works of Albert Shiryaev on Markov Decision Processes and Some Later Developments
Eugene A. Feinberg
https://arxiv.org/abs/2506.04896 https:/…

On Pioneering Works of Albert Shiryaev on Markov Decision Processes and Some Later Developments
This article is dedicated to three fundamental papers on Markov Decision Processes and on control with incomplete observations published by Albert Shiryaev approximately sixty years ago. One of these papers was coauthored with O.V. Viskov. We discuss some of the results and some of many rich ideas presented in these papers and survey some later developments. At the end we mention some recent studies of Albert Shiryaev on Kolmogorov's equations for jump Markov processes and on control of continu…

The space of multifurcating ranked tree shapes: enumeration, lattice structure, and Markov chains
Julie Zhang, Noah A. Rosenberg, Julia A. Palacios
https://arxiv.org/abs/2506.10856

The space of multifurcating ranked tree shapes: enumeration, lattice structure, and Markov chains
Coalescent models of bifurcating genealogies are used to infer evolutionary parameters from molecular data. However, there are many situations where bifurcating genealogies do not accurately reflect the true underlying ancestral history of samples, and a multifurcating genealogy is required. The space of multifurcating genealogical trees, where nodes can have more than two descendants, is largely underexplored in the setting of coalescent inference. In this paper, we examine the space of rooted…

This https://arxiv.org/abs/2505.18300 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csLG_…

Beyond Self-Repellent Kernels: History-Driven Target Towards Efficient Nonlinear MCMC on General Graphs
We propose a history-driven target (HDT) framework in Markov Chain Monte Carlo (MCMC) to improve any random walk algorithm on discrete state spaces, such as general undirected graphs, for efficient sampling from target distribution $\boldsymbolμ$. With broad applications in network science and distributed optimization, recent innovations like the self-repellent random walk (SRRW) achieve near-zero variance by prioritizing under-sampled states through transition kernel modifications based on pa…

Law of the Iterated Logarithm for Markov Semigroups with Exponential Mixing in the Wasserstein Distance
Dawid Czapla, Sander C. Hille, Katarzyna Horbacz, Hanna Wojew\'odka-\'Sci\k{a}\.zko
https://arxiv.org/abs/2507.01144

Law of the Iterated Logarithm for Markov Semigroups with Exponential Mixing in the Wasserstein Distance
In this paper, we establish the law of the iterated logarithm for a wide class of non-stationary, continuous-time Markov processes evolving on Polish spaces. Specifically, our results apply to additive functionals of processes governed by stochastically continuous Markov-Feller semigroups that exhibit exponential mixing and non-expansiveness in the Wasserstein norm, provided that a suitable moment condition on the initial distributions is satisfied.