Tootfinder

Autonomous vehicle decision and control through reinforcement learning with traffic flow randomization
Yuan Lin, Antai Xie, Xiao Liu
https://arxiv.org/abs/2403.02882

Autonomous vehicle decision and control through reinforcement learning with traffic flow randomization
Most of the current studies on autonomous vehicle decision-making and control tasks based on reinforcement learning are conducted in simulated environments. The training and testing of these studies are carried out under rule-based microscopic traffic flow, with little consideration of migrating them to real or near-real environments to test their performance. It may lead to a degradation in performance when the trained model is tested in more realistic traffic scenes. In this study, we propose…

How activists blur the lines between birth control and abortion
The national debate over #IVF, unfolding after an #Alabama court decision prompted multiple clinics in the state to halt operations, prompts a question:
What might be next?
Could other fertility treatments and even birth control be under thr…

How activists blur the lines between birth control and abortion
Contraception, like IVF, poses problems for those claiming personhood begins at conception.

This https://arxiv.org/abs/2204.01410 has been replaced.
link: https://scholar.google.com/scholar?q=a

Ergodic control of a heterogeneous population and application to electricity pricing
We consider a control problem for a heterogeneous population composed of agents able to switch at any time between different options. The controller aims to maximize an average gain per time unit, supposing that the population is of infinite size. This leads to an ergodic control problem for a "mean-field" Markov Decision Process in which the state space is a product of simplices, and the population evolves according to controlled linear dynamics. By exploiting contraction properties of the dyn…

Autonomous vehicle decision and control through reinforcement learning with traffic flow randomization
Yuan Lin, Antai Xie, Xiao Liu
https://arxiv.org/abs/2403.02882

Autonomous vehicle decision and control through reinforcement learning with traffic flow randomization
Most of the current studies on autonomous vehicle decision-making and control tasks based on reinforcement learning are conducted in simulated environments. The training and testing of these studies are carried out under rule-based microscopic traffic flow, with little consideration of migrating them to real or near-real environments to test their performance. It may lead to a degradation in performance when the trained model is tested in more realistic traffic scenes. In this study, we propose…

Safe Hybrid-Action Reinforcement Learning-Based Decision and Control for Discretionary Lane Change
Ruichen Xu, Xiao Liu, Jinming Xu, Yuan Lin
https://arxiv.org/abs/2403.00446

Safe Hybrid-Action Reinforcement Learning-Based Decision and Control for Discretionary Lane Change
Autonomous lane-change, a key feature of advanced driver-assistance systems, can enhance traffic efficiency and reduce the incidence of accidents. However, safe driving of autonomous vehicles remains challenging in complex environments. How to perform safe and appropriate lane change is a popular topic of research in the field of autonomous driving. Currently, few papers consider the safety of reinforcement learning in autonomous lane-change scenarios. We introduce safe hybrid-action reinforcem…

Collaborative Control Method of Transit Signal Priority Based on Cooperative Game and Reinforcement Learning
Hao Qin, Weishi Zhang
https://arxiv.org/abs/2404.19683

Collaborative Control Method of Transit Signal Priority Based on Cooperative Game and Reinforcement Learning
To address the low efficiency in priority signal control within intelligent transportation systems, this study introduces a novel eight-phase priority signal control method, CBQL-TSP, leveraging a hybrid decision-making framework that integrates cooperative game theory and reinforcement learning. This approach conceptualizes the allocation of bus signal priorities as a multi-objective decision-making problem across an eight-phase signal sequence, differentiating between priority and non-priorit…

This https://arxiv.org/abs/2402.17524 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Highway Discretionary Lane-change Decision and Control Using Model Predictive Control
To enable autonomous vehicles to perform discretionary lane change amidst the random traffic flow on highways, this paper introduces a decision-making and control method for vehicle lane change based on Model Predictive Control (MPC). This approach divides the driving control of vehicles on highways into two parts: lane-change decision and lane-change control, both of which are solved using the MPC method. In the lanechange decision module, the minimum driving costs for each lane are computed a…

Predicting Fairness of ML Software Configuration
Salvador Robles Herrera, Verya Monjezi, Vladik Kreinovich, Ashutosh Trivedi, Saeid Tizpaz-Niari
https://arxiv.org/abs/2404.19100 <…

Predicting Fairness of ML Software Configuration
This paper investigates the relationships between hyperparameters of machine learning and fairness. Data-driven solutions are increasingly used in critical socio-technical applications where ensuring fairness is important. Rather than explicitly encoding decision logic via control and data structures, the ML developers provide input data, perform some pre-processing, choose ML algorithms, and tune hyperparameters (HPs) to infer a program that encodes the decision logic. Prior works report that …

This https://arxiv.org/abs/2402.07946 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csHC_…

Re-Envisioning Command and Control
Future warfare will require Command and Control (C2) decision-making to occur in more complex, fast-paced, ill-structured, and demanding conditions. C2 will be further complicated by operational challenges such as Denied, Degraded, Intermittent, and Limited (DDIL) communications and the need to account for many data streams, potentially across multiple domains of operation. Yet, current C2 practices -- which stem from the industrial era rather than the emerging intelligence era -- are linear an…

Hillary Clinton is warning about the legality of birth control in the wake of a decision by the Alabama Supreme Court that found frozen embryos created through fertility treatments are children under state law.
“They came for abortion first. Now it’s [in vitro fertilization], and next it’ll be birth control,” the 2016 Democratic presidential nominee and secretary of State said in a post on X, the platform formerly known as Twitter.
“The extreme right won’t stop trying to exert go…

Hillary Clinton warns birth control is ‘next’ after Alabama IVF ruling
Hillary Clinton is warning about the legality of birth control in the wake of a decision by the Alabama Supreme Court that found frozen embryos created through fertility treatments are children under state law. “They came for abortion first. Now it’s [in vitro fertilization] and next it’ll be birth control,” the 2016 Democratic presidential nominee…

Deep reinforcement learning of airfoil pitch control in a highly disturbed environment using partial observations
Diederik Beckers, Jeff D. Eldredge
https://arxiv.org/abs/2404.01506

Deep reinforcement learning of airfoil pitch control in a highly disturbed environment using partial observations
This study explores the application of deep reinforcement learning (RL) to design an airfoil pitch controller capable of minimizing lift variations in randomly disturbed flows. The controller, treated as an agent in a partially observable Markov decision process, receives non-Markovian observations from the environment, simulating practical constraints where flow information is limited to force and pressure sensors. Deep RL, particularly the TD3 algorithm, is used to approximate an optimal cont…

Go Beyond Black-box Policies: Rethinking the Design of Learning Agent for Interpretable and Verifiable HVAC Control
Zhiyu An, Xianzhong Ding, Wan Du
https://arxiv.org/abs/2403.00172

Go Beyond Black-box Policies: Rethinking the Design of Learning Agent for Interpretable and Verifiable HVAC Control
Recent research has shown the potential of Model-based Reinforcement Learning (MBRL) to enhance energy efficiency of Heating, Ventilation, and Air Conditioning (HVAC) systems. However, existing methods rely on black-box thermal dynamics models and stochastic optimizers, lacking reliability guarantees and posing risks to occupant health. In this work, we overcome the reliability bottleneck by redesigning HVAC controllers using decision trees extracted from existing thermal dynamics models and hi…

ML-based handover prediction over a real O-RAN deployment using RAN Intelligent controller
Merim Dzaferagic, Bruno Missi Xavier, Diarmuid Collins, Vince D'Onofrio, Magnos Martinello, Marco Ruffini
https://arxiv.org/abs/2404.19671

ML-based handover prediction over a real O-RAN deployment using RAN Intelligent controller
O-RAN introduces intelligent and flexible network control in all parts of the network. The use of controllers with open interfaces allow us to gather real time network measurements and make intelligent/informed decision. The work in this paper focuses on developing a use-case for open and reconfigurable networks to investigate the possibility to predict handover events and understand the value of such predictions for all stakeholders that rely on the communication network to conduct their busin…

Quantum Markov Decision Processes Part I: General Theory, Approximations, and Classes of Policies
Naci Saldi, Sina Sanjari, Serdar Yuksel
https://arxiv.org/abs/2402.14649

Quantum Markov Decision Processes Part I: General Theory, Approximations, and Classes of Policies
In this two part article, the aim is to develop a quantum counterpart to classical Markov decision processes (MDPs). In Part I, we provide a very general formulation of quantum MDPs with state and action spaces in the quantum domain, quantum transitions, and cost functions. Once we formulate the quantum MDP (q-MDP), our focus shifts to establishing the verification theorem that proves the sufficiency of Markovian quantum control policies and provides a dynamic programming principle. Subsequentl…

Decision Transformer as a Foundation Model for Partially Observable Continuous Control
Xiangyuan Zhang, Weichao Mao, Haoran Qiu, Tamer Ba\c{s}ar
https://arxiv.org/abs/2404.02407 <…

Decision Transformer as a Foundation Model for Partially Observable Continuous Control
Closed-loop control of nonlinear dynamical systems with partial-state observability demands expert knowledge of a diverse, less standardized set of theoretical tools. Moreover, it requires a delicate integration of controller and estimator designs to achieve the desired system behavior. To establish a general controller synthesis framework, we explore the Decision Transformer (DT) architecture. Specifically, we first frame the control task as predicting the current optimal action based on past …

This https://arxiv.org/abs/2310.18027 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_sta…

Bayesian Prognostic Covariate Adjustment With Additive Mixture Priors
Effective and rapid decision-making from randomized controlled trials (RCTs) requires unbiased and precise treatment effect inferences. Two strategies to address this requirement are to adjust for covariates that are highly correlated with the outcome, and to leverage historical control information via Bayes' theorem. We propose a new Bayesian prognostic covariate adjustment methodology, referred to as Bayesian PROCOVA, that combines these two strategies. Covariate adjustment in Bayesian PROCOV…

Open Datasets for Satellite Radio Resource Control
Husnain Shahid, Miguel \'A. V\'azquez, Musbah Shaat, Pol Henarejos, Nuria T Quijada, Enrique Marinu
https://arxiv.org/abs/2404.13920

Open Datasets for Satellite Radio Resource Control
In Non-Terrestrial Networks (NTN), achieving effective radio resource allocation across multi-satellite system, encompassing efficient channel and bandwidth allocation, effective beam management, power control and interference mitigation, poses significant challenges due to the varying satellite links and highly dynamic nature of user traffic. This calls for the development of an intelligent decision-making controller using Artificial Intelligence (AI) to efficiently manage resources in this co…

Foundations of Value of Information: A Semantic Metric for Networked Control Systems Tasks
Touraj Soleymani, John S. Baras, Sandra Hirche, Karl H. Johansson
https://arxiv.org/abs/2403.11927

Foundations of Value of Information: A Semantic Metric for Networked Control Systems Tasks
In this chapter, we present our recent invention, i.e., the notion of the value of information$\unicode{x2014}$a semantic metric that is fundamental for networked control systems tasks. We begin our analysis by formulating a causal tradeoff between the packet rate and the regulation cost, with an encoder and a decoder as two distributed decision makers, and show that the valuation of information is conceivable and quantifiable grounded on this tradeoff. More precisely, we characterize an equili…

Highway Discretionary Lane-change Decision and Control Using Model Predictive Control
Zishun Zheng, Xiao Liu, Yuan Lin
https://arxiv.org/abs/2402.17524 htt…

Highway Discretionary Lane-change Decision and Control Using Model Predictive Control
To enable vehicles to perform automatic lane change amidst the random traffic flow on highways, this paper introduces a decision-making and control method for vehicle lane-change based on Model Predictive Control (MPC). This approach divides the driving control of vehicles on highways into two parts: lane-change decision and lane-change control, both of which are solved using the MPC method. In the lane-change decision module, the minimum driving costs for each lane are computed and compared by…

This https://arxiv.org/abs/2312.16044 has been replaced.
link: https://scholar.google.com/scholar?q=a

LLMLight: Large Language Models as Traffic Signal Control Agents
Traffic Signal Control (TSC) is a crucial component in urban traffic management, aiming to optimize road network efficiency and reduce congestion. Traditional methods in TSC, primarily based on transportation engineering and reinforcement learning (RL), often exhibit limitations in generalization across varied traffic scenarios and lack interpretability. This paper presents LLMLight, a novel framework employing Large Language Models (LLMs) as decision-making agents for TSC. Specifically, the fr…

Supervisory Control Theory with Event Forcing
Michel Reniers, Kai Cai
https://arxiv.org/abs/2404.08469 https://arxiv.org/pdf/2404.084…

Supervisory Control Theory with Event Forcing
In the Ramadge-Wonham supervisory control theory the only interaction mechanism between supervisor and plant is that the supervisor may enable/disable events from the plant and the plant makes a final decision about which of the enabled events is actually taking place. In this paper, the interaction between supervisor and plant is enriched by allowing the supervisor to force specific events (called forcible events) that are allowed to preempt uncontrollable events. A notion of forcible-controll…

A Stochastic Model-Based Control Methodology for Glycemic Management in the Intensive Care Unit
Melike Sirlanci, George Hripcsak, Cecilia C. Low Wang, J. N. Stroh, Yanran Wang, Tellen D. Bennett, Andrew M. Stuart, David J. Albers
https://arxiv.org/abs/2403.14934

A Stochastic Model-Based Control Methodology for Glycemic Management in the Intensive Care Unit
Intensive care unit (ICU) patients exhibit erratic blood glucose (BG) fluctuations, including hypoglycemic and hyperglycemic episodes, and require exogenous insulin delivery to keep their BG in healthy ranges. Glycemic control via glycemic management (GM) is associated with reduced mortality and morbidity in the ICU, but GM increases the cognitive load on clinicians. The availability of robust, accurate, and actionable clinical decision support (CDS) tools reduces this burden and assists in the…

Fast Decision Algorithms for Efficient Access Point Assignment in SDN-Controlled Wireless Access Networks
Pablo Fondo-Ferreiro, Saber Mhiri, Cristina L\'opez-Bravo, Francisco Javier Gonz\'alez-Casta\~no, Felipe Gil-Casti\~neira
https://arxiv.org/abs/2403.18745

Fast Decision Algorithms for Efficient Access Point Assignment in SDN-Controlled Wireless Access Networks
Global optimization of access point (AP) assignment to user terminals requires efficient monitoring of user behavior, fast decision algorithms, efficient control signaling, and fast AP reassignment mechanisms. In this scenario, software defined networking (SDN) technology may be suitable for network monitoring, signaling, and control. We recently proposed embedding virtual switches in user terminals for direct management by an SDN controller, further contributing to SDN-oriented access network …

This https://arxiv.org/abs/2404.10849 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csRO_…

End-To-End Training and Testing Gamification Framework to Learn Human Highway Driving
The current autonomous stack is well modularized and consists of perception, decision making and control in a handcrafted framework. With the advances in artificial intelligence (AI) and computing resources, researchers have been pushing the development of end-to-end AI for autonomous driving, at least in problems of small searching space such as in highway scenarios, and more and more photorealistic simulation will be critical for efficient learning. In this research, we propose a novel game-b…

This https://arxiv.org/abs/2212.12561 has been replaced.
link: https://scholar.google.com/scholar?q=a

An active learning method for solving competitive multi-agent decision-making and control problems
To identify a stationary action profile for a population of competitive agents, each executing private strategies, we introduce a novel active-learning scheme where a centralized external observer (or entity) can probe the agents' reactions and recursively update simple local parametric estimates of the action-reaction mappings. Under very general working assumptions (not even assuming that a stationary profile exists), sufficient conditions are established to assess the asymptotic properties o…

Re-Envisioning Command and Control
Kaleb McDowell, Ellen Novoseller, Anna Madison, Vinicius G. Goecks, Christopher Kelshaw
https://arxiv.org/abs/2402.07946

Re-Envisioning Command and Control
Future warfare will require Command and Control (C2) decision-making to occur in more complex, fast-paced, ill-structured, and demanding conditions. C2 will be further complicated by operational challenges such as Denied, Degraded, Intermittent, and Limited (DDIL) communications and the need to account for many data streams, potentially across multiple domains of operation. Yet, current C2 practices -- which stem from the industrial era rather than the emerging intelligence era -- are linear an…

Stackelberg reinsurance and premium decisions with MV criterion and irreversibility
Zongxia Liang, Xiaodong Luo
https://arxiv.org/abs/2402.11580 https://…

Stackelberg reinsurance and premium decisions with MV criterion and irreversibility
We study a reinsurance Stackelberg game in which both the insurer and the reinsurer adopt the mean-variance (abbr. MV) criterion in their decision-making and the reinsurance is irreversible. We apply a unified singular control framework where irreversible reinsurance contracts can be signed in both discrete and continuous times. The results theoretically illustrate that, rather than continuous-time contracts or a bunch of discrete-time contracts, a single once-for-all reinsurance contract is pr…

This https://arxiv.org/abs/2401.14029 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_mat…

Towards a Systems Theory of Algorithms
Traditionally, numerical algorithms are seen as isolated pieces of code confined to an {\em in silico} existence. However, this perspective is not appropriate for many modern computational approaches in control, learning, or optimization, wherein {\em in vivo} algorithms interact with their environment. Examples of such {\em open algorithms} include various real-time optimization-based control strategies, reinforcement learning, decision-making architectures, online optimization, and many more.…

This https://arxiv.org/abs/2312.16044 has been replaced.
link: https://scholar.google.com/scholar?q=a

LLMLight: Large Language Models as Traffic Signal Control Agents
Traffic Signal Control (TSC) is a crucial component in urban traffic management, aiming to optimize road network efficiency and reduce congestion. Traditional methods in TSC, primarily based on transportation engineering and reinforcement learning (RL), often exhibit limitations in generalization across varied traffic scenarios and lack interpretability. This paper presents LLMLight, a novel framework employing Large Language Models (LLMs) as decision-making agents for TSC. Specifically, the fr…

This https://arxiv.org/abs/2308.01152 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDM_…

Skolem Meets Bateman-Horn
The Skolem Problem asks to determine whether a given integer linear recurrence sequence has a zero term. This problem arises across a wide range of topics in computer science, including loop termination, formal languages, automata theory, and control theory, amongst many others. Decidability of the Skolem Problem is notoriously open. The state of the art is a decision procedure for recurrences of order at most 4: an advance achieved some 40 years ago, based on Baker's theorem on linear forms in…

This https://arxiv.org/abs/2402.05388 has been replaced.
link: https://scholar.google.com/scholar?q=a

Form-From: A Design Space of Social Media Systems
Social media systems are as varied as they are pervasive. They have been almost universally adopted for a broad range of purposes including work, entertainment, activism, and decision making. As a result, they have also diversified, with many distinct designs differing in content type, organization, delivery mechanism, access control, and many other dimensions. In this work, we aim to characterize and then distill a concise design space of social media systems that can help us understand simila…

A note on weak compactness of occupation measures for an absorbing Markov decision process
Fran\c{c}ois Dufour, Tom\'as Prieto-Rumeau
https://arxiv.org/abs/2402.10672

A note on weak compactness of occupation measures for an absorbing Markov decision process
We consider an absorbing Markov decision process with a discrete time parameter with Borel state and action spaces. We prove that the set of occupation measures is compact for the weak topology if and only if the control model is uniformly absorbing.

This https://arxiv.org/abs/2403.01450 has been replaced.
link: https://scholar.google.com/scholar?q=a

Collision-Free Robot Navigation in Crowded Environments using Learning based Convex Model Predictive Control
Navigating robots safely and efficiently in crowded and complex environments remains a significant challenge. However, due to the dynamic and intricate nature of these settings, planning efficient and collision-free paths for robots to track is particularly difficult. In this paper, we uniquely bridge the robot's perception, decision-making and control processes by utilizing the convex obstacle-free region computed from 2D LiDAR data. The overall pipeline is threefold: (1) We proposes a robot n…

Data-Adaptive Tradeoffs among Multiple Risks in Distribution-Free Prediction
Drew T. Nguyen, Reese Pathak, Anastasios N. Angelopoulos, Stephen Bates, Michael I. Jordan
https://arxiv.org/abs/2403.19605

Data-Adaptive Tradeoffs among Multiple Risks in Distribution-Free Prediction
Decision-making pipelines are generally characterized by tradeoffs among various risk functions. It is often desirable to manage such tradeoffs in a data-adaptive manner. As we demonstrate, if this is done naively, state-of-the art uncertainty quantification methods can lead to significant violations of putative risk guarantees. To address this issue, we develop methods that permit valid control of risk when threshold and tradeoff parameters are chosen adaptively. Our methodology supports mon…

Model-Based Reinforcement Learning Control of Reaction-Diffusion Problems
Christina Schenk, Aditya Vasudevan, Maciej Haranczyk, Ignacio Romero
https://arxiv.org/abs/2402.14446

Model-Based Reinforcement Learning Control of Reaction-Diffusion Problems
Mathematical and computational tools have proven to be reliable in decision-making processes. In recent times, in particular, machine learning-based methods are becoming increasingly popular as advanced support tools. When dealing with control problems, reinforcement learning has been applied to decision-making in several applications, most notably in games. The success of these methods in finding solutions to complex problems motivates the exploration of new areas where they can be employed to…

Engineering consensus in static networks with unknown disruptors
Agathe Bouis, Christopher Lowe, Ruaridh A. Clark, Malcolm Macdonald
https://arxiv.org/abs/2403.05272

Engineering consensus in static networks with unknown disruptors
Distributed control increases system scalability, flexibility, and redundancy. Foundational to such decentralisation is consensus formation, by which decision-making and coordination are achieved. However, decentralised multi-agent systems are inherently vulnerable to disruption. To develop a resilient consensus approach, inspiration is taken from the study of social systems and their dynamics; specifically, the Deffuant Model. A dynamic algorithm is presented enabling efficient consensus to be…

Optimal Control Synthesis of Markov Decision Processes for Efficiency with Surveillance Tasks
Yu Chen, Xuanyuan Yin, Shaoyuan Li, Xiang Yin
https://arxiv.org/abs/2403.18632

Optimal Control Synthesis of Markov Decision Processes for Efficiency with Surveillance Tasks
We investigate the problem of optimal control synthesis for Markov Decision Processes (MDPs), addressing both qualitative and quantitative objectives. Specifically, we require the system to fulfill a qualitative surveillance task in the sense that a specific region of interest can be visited infinitely often with probability one. Furthermore, to quantify the performance of the system, we consider the concept of efficiency, which is defined as the ratio between rewards and costs. This measure is…

Strategizing against Q-learners: A Control-theoretical Approach
Yuksel Arslantas, Ege Yuceel, Muhammed O. Sayin
https://arxiv.org/abs/2403.08906 https://…

Strategizing against Q-learners: A Control-theoretical Approach
In this paper, we explore the susceptibility of the Q-learning algorithm (a classical and widely used reinforcement learning method) to strategic manipulation of sophisticated opponents in games. We quantify how much a strategically sophisticated agent can exploit a naive Q-learner if she knows the opponent's Q-learning algorithm. To this end, we formulate the strategic actor's problem as a Markov decision process (with a continuum state space encompassing all possible Q-values) as if the Q-lea…

This https://arxiv.org/abs/2404.05564 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csNI_…

Optimal Flow Admission Control in Edge Computing via Safe Reinforcement Learning
With the uptake of intelligent data-driven applications, edge computing infrastructures necessitate a new generation of admission control algorithms to maximize system performance under limited and highly heterogeneous resources. In this paper, we study how to optimally select information flows which belong to different classes and dispatch them to multiple edge servers where deployed applications perform flow analytic tasks. The optimal policy is obtained via constrained Markov decision proces…

Confidence-Aware Decision-Making and Control for Tool Selection
Ajith Anil Meera, Pablo Lanillos
https://arxiv.org/abs/2403.03808 https://

Confidence-Aware Decision-Making and Control for Tool Selection
Self-reflecting about our performance (e.g., how confident we are) before doing a task is essential for decision making, such as selecting the most suitable tool or choosing the best route to drive. While this form of awareness -- thinking about our performance or metacognitive performance -- is well-known in humans, robots still lack this cognitive ability. This reflective monitoring can enhance their embodied decision power, robustness and safety. Here, we take a step in this direction by int…

Data-Adaptive Tradeoffs among Multiple Risks in Distribution-Free Prediction
Drew T. Nguyen, Reese Pathak, Anastasios N. Angelopoulos, Stephen Bates, Michael I. Jordan
https://arxiv.org/abs/2403.19605

Data-Adaptive Tradeoffs among Multiple Risks in Distribution-Free Prediction
Decision-making pipelines are generally characterized by tradeoffs among various risk functions. It is often desirable to manage such tradeoffs in a data-adaptive manner. As we demonstrate, if this is done naively, state-of-the art uncertainty quantification methods can lead to significant violations of putative risk guarantees. To address this issue, we develop methods that permit valid control of risk when threshold and tradeoff parameters are chosen adaptively. Our methodology supports mon…

This https://arxiv.org/abs/2402.05388 has been replaced.
link: https://scholar.google.com/scholar?q=a

Form-From: A Design Space of Social Media Systems
Social media systems are as varied as they are pervasive. They have been almost universally adopted for a broad range of purposes including work, entertainment, activism, and decision making. As a result, they have also diversified, with many distinct designs differing in content type, organization, delivery mechanism, access control, and many other dimensions. In this work, we aim to characterize and then distill a concise design space of social media systems that can help us understand simila…

Model Free Deep Deterministic Policy Gradient Controller for Setpoint Tracking of Non-minimum Phase Systems
Fatemeh Tavakkoli, Pouria Sarhadi, Benoit Clement, Wasif Naeem
https://arxiv.org/abs/2402.17703

Model Free Deep Deterministic Policy Gradient Controller for Setpoint Tracking of Non-minimum Phase Systems
Deep Reinforcement Learning (DRL) techniques have received significant attention in control and decision-making algorithms. Most applications involve complex decision-making systems, justified by the algorithms' computational power and cost. While model-based versions are emerging, model-free DRL approaches are intriguing for their independence from models, yet they remain relatively less explored in terms of performance, particularly in applied control. This study conducts a thorough performan…

This https://arxiv.org/abs/2104.14662 has been replaced.
link: https://scholar.google.com/scholar?q=a

Dynamic Population Games: A Tractable Intersection of Mean-Field Games and Population Games
In many real-world large-scale decision problems, self-interested agents have individual dynamics and optimize their own long-term payoffs. Important examples include the competitive access to shared resources (e.g., roads, energy, or bandwidth) but also non-engineering domains like epidemic propagation and control. These problems are natural to model as mean-field games. However, existing mathematical formulations of mean field games have had limited applicability in practice, since they requi…

Bayesian Optimization with Adaptive Kernels for Robot Control
Ruben Martinez-Cantin
https://arxiv.org/abs/2402.07021 https://arxiv.or…

Bayesian Optimization with Adaptive Kernels for Robot Control
Active policy search combines the trial-and-error methodology from policy search with Bayesian optimization to actively find the optimal policy. First, policy search is a type of reinforcement learning which has become very popular for robot control, for its ability to deal with complex continuous state and action spaces. Second, Bayesian optimization is a sample efficient global optimization method that uses a surrogate model, like a Gaussian process, and optimal decision making to carefully s…

Parking of Connected Automated Vehicles: Vehicle Control, Parking Assignment, and Multi-agent Simulation
Xu Shen, Yongkeun Choi, Alex Wong, Francesco Borrelli, Scott Moura, Soomin Woo
https://arxiv.org/abs/2402.14183

Parking of Connected Automated Vehicles: Vehicle Control, Parking Assignment, and Multi-agent Simulation
This paper introduces a novel approach to optimize the parking efficiency for fleets of Connected and Automated Vehicles (CAVs). We present a novel multi-vehicle parking simulator, equipped with hierarchical path planning and collision avoidance capabilities for individual CAVs. The simulator is designed to capture the key decision-making processes in parking, from low-level vehicle control to high-level parking assignment, and it enables the effective assessment of parking strategies for large…

Demystifying Deep Reinforcement Learning-Based Autonomous Vehicle Decision-Making
Hanxi Wan, Pei Li, Arpan Kusari
https://arxiv.org/abs/2403.11432 https://…

Demystifying Deep Reinforcement Learning-Based Autonomous Vehicle Decision-Making
With the advent of universal function approximators in the domain of reinforcement learning, the number of practical applications leveraging deep reinforcement learning (DRL) has exploded. Decision-making in automated driving tasks has emerged as a chief application among them, taking the sensor data or the higher-order kinematic variables as the input and providing a discrete choice or continuous control output. However, the black-box nature of the models presents an overwhelming limitation th…

Engineering consensus in static networks with unknown disruptors
Agathe Bouis, Christopher Lowe, Ruaridh A. Clark, Malcolm Macdonald
https://arxiv.org/abs/2403.05272

Engineering consensus in static networks with unknown disruptors
Distributed control increases system scalability, flexibility, and redundancy. Foundational to such decentralisation is consensus formation, by which decision-making and coordination are achieved. However, decentralised multi-agent systems are inherently vulnerable to disruption. To develop a resilient consensus approach, inspiration is taken from the study of social systems and their dynamics; specifically, the Deffuant Model. A dynamic algorithm is presented enabling efficient consensus to be…

Demystifying Deep Reinforcement Learning-Based Autonomous Vehicle Decision-Making
Hanxi Wan, Pei Li, Arpan Kusari
https://arxiv.org/abs/2403.11432 https://…

Demystifying Deep Reinforcement Learning-Based Autonomous Vehicle Decision-Making
With the advent of universal function approximators in the domain of reinforcement learning, the number of practical applications leveraging deep reinforcement learning (DRL) has exploded. Decision-making in automated driving tasks has emerged as a chief application among them, taking the sensor data or the higher-order kinematic variables as the input and providing a discrete choice or continuous control output. However, the black-box nature of the models presents an overwhelming limitation th…

Optimal Flow Admission Control in Edge Computing via Safe Reinforcement Learning
A. Fox (Avignon Universit\'e), F. De Pellegrini (Avignon Universit\'e), F. Faticanti (ENS Lyon), E. Altman (Inria), F. Bronzino (ENS Lyon)
https://arxiv.org/abs/2404.05564

Optimal Flow Admission Control in Edge Computing via Safe Reinforcement Learning
With the uptake of intelligent data-driven applications, edge computing infrastructures necessitate a new generation of admission control algorithms to maximize system performance under limited and highly heterogeneous resources. In this paper, we study how to optimally select information flows which belong to different classes and dispatch them to multiple edge servers where deployed applications perform flow analytic tasks. The optimal policy is obtained via constrained Markov decision proces…

This https://arxiv.org/abs/2403.02882 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Autonomous vehicle decision and control through reinforcement learning with traffic flow randomization
Most of the current studies on autonomous vehicle decision-making and control tasks based on reinforcement learning are conducted in simulated environments. The training and testing of these studies are carried out under rule-based microscopic traffic flow, with little consideration of migrating them to real or near-real environments to test their performance. It may lead to a degradation in performance when the trained model is tested in more realistic traffic scenes. In this study, we propose…

Improved Algebraic Inverter Modelling for Four-Wire Power Flow Optimization
Rahmat Heidari, Frederik Geth
https://arxiv.org/abs/2403.07285 https://<…

Improved Algebraic Inverter Modelling for Four-Wire Power Flow Optimization
This paper discusses the modeling of inverters used in distributed energy resources in steady state. Modeling the interaction between distribution grids and inverter-based resources is crucial to understand the consequences for the network's operational and planning processes. This work highlights the limitations of existing models and emphasizes the need for better representations of inverters and their control laws in decision-making contexts. Improved steady-state grid-following and grid-for…

Improved Algebraic Inverter Modelling for Four-Wire Power Flow Optimization
Rahmat Heidari, Frederik Geth
https://arxiv.org/abs/2403.07285 https://<…

Improved Algebraic Inverter Modelling for Four-Wire Power Flow Optimization
This paper discusses the modeling of inverters used in distributed energy resources in steady state. Modeling the interaction between distribution grids and inverter-based resources is crucial to understand the consequences for the network's operational and planning processes. This work highlights the limitations of existing models and emphasizes the need for better representations of inverters and their control laws in decision-making contexts. Improved steady-state grid-following and grid-for…

Intelligence and Motion Models of Continuum Robots: an Overview
Oxana Shamilyan, Ievgen Kabin, Zoya Dyka, Oleksandr Sudakov, Andrii Cherninskyi, Marcin Brzozowski, Peter Langendoerfer
https://arxiv.org/abs/2404.06171

Intelligence and Motion Models of Continuum Robots: an Overview
Many technical solutions are bio-inspired. Octopus-inspired robotic arms belong to continuum robots which are used in minimally invasive surgery or for technical system restoration in areas difficult-toaccess. Continuum robot missions are bounded with their motions, whereby the motion of the robots is controlled by humans via wireless communication. In case of a lost connection, robot autonomy is required. Distributed control and distributed decision-making mechanisms based on artificial intell…

A code-driven tutorial on encrypted control: From pioneering realizations to modern implementations
Nils Schl\"uter, Junsoo Kim, Moritz Schulze Darup
https://arxiv.org/abs/2404.04727

A code-driven tutorial on encrypted control: From pioneering realizations to modern implementations
The growing interconnectivity in control systems due to robust wireless communication and cloud usage paves the way for exciting new opportunities such as data-driven control and service-based decision-making. At the same time, connected systems are susceptible to cyberattacks and data leakages. Against this background, encrypted control aims to increase the security and safety of cyber-physical systems. A central goal is to ensure confidentiality of process data during networked controller eva…

3D Object Visibility Prediction in Autonomous Driving
Chuanyu Luo, Nuo Cheng, Ren Zhong, Haipeng Jiang, Wenyu Chen, Aoli Wang, Pu Li
https://arxiv.org/abs/2403.03681

3D Object Visibility Prediction in Autonomous Driving
With the rapid advancement of hardware and software technologies, research in autonomous driving has seen significant growth. The prevailing framework for multi-sensor autonomous driving encompasses sensor installation, perception, path planning, decision-making, and motion control. At the perception phase, a common approach involves utilizing neural networks to infer 3D bounding box (Bbox) attributes from raw sensor data, including classification, size, and orientation. In this paper, we prese…

This https://arxiv.org/abs/2306.02574 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Bayesian Learning of Optimal Policies in Markov Decision Processes with Countably Infinite State-Space
Models of many real-life applications, such as queuing models of communication networks or computing systems, have a countably infinite state-space. Algorithmic and learning procedures that have been developed to produce optimal policies mainly focus on finite state settings, and do not directly apply to these models. To overcome this lacuna, in this work we study the problem of optimal control of a family of discrete-time countable state-space Markov Decision Processes (MDPs) governed by an un…

This https://arxiv.org/abs/2209.08349 has been replaced.
link: https://scholar.google.com/scholar?q=a

Unguided Self-exploration in Narrow Spaces with Safety Region Enhanced Reinforcement Learning for Ackermann-steering Robots
In narrow spaces, motion planning based on the traditional hierarchical autonomous system could cause collisions due to mapping, localization, and control noises, especially for car-like Ackermann-steering robots which suffer from non-convex and non-holonomic kinematics. To tackle these problems, we leverage deep reinforcement learning which is verified to be effective in self-decision-making, to self-explore in narrow spaces without a given map and destination while avoiding collisions. Specif…

Multi-Agent Soft Actor-Critic with Global Loss for Autonomous Mobility-on-Demand Fleet Control
Zeno Woywood, Jasper I. Wiltfang, Julius Luy, Tobias Enders, Maximilian Schiffer
https://arxiv.org/abs/2404.06975

Multi-Agent Soft Actor-Critic with Global Loss for Autonomous Mobility-on-Demand Fleet Control
We study a sequential decision-making problem for a profit-maximizing operator of an Autonomous Mobility-on-Demand system. Optimizing a central operator's vehicle-to-request dispatching policy requires efficient and effective fleet control strategies. To this end, we employ a multi-agent Soft Actor-Critic algorithm combined with weighted bipartite matching. We propose a novel vehicle-based algorithm architecture and adapt the critic's loss function to appropriately consider global actions. Furt…

Enhancing High-Speed Cruising Performance of Autonomous Vehicles through Integrated Deep Reinforcement Learning Framework
Jinhao Liang, Kaidi Yang, Chaopeng Tan, Jinxiang Wang, Guodong Yin
https://arxiv.org/abs/2404.14713

Enhancing High-Speed Cruising Performance of Autonomous Vehicles through Integrated Deep Reinforcement Learning Framework
High-speed cruising scenarios with mixed traffic greatly challenge the road safety of autonomous vehicles (AVs). Unlike existing works that only look at fundamental modules in isolation, this work enhances AV safety in mixed-traffic high-speed cruising scenarios by proposing an integrated framework that synthesizes three fundamental modules, i.e., behavioral decision-making, path-planning, and motion-control modules. Considering that the integrated framework would increase the system complexity…

This https://arxiv.org/abs/2301.05089 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Approximate Information States for Worst-Case Control and Learning in Uncertain Systems
In this paper, we investigate discrete-time decision-making problems in uncertain systems with partially observed states. We consider a non-stochastic model, where uncontrolled disturbances acting on the system take values in bounded sets with unknown distributions. We present a general framework for decision-making in such problems by using the notion of the information state and approximate information state, and introduce conditions to identify an uncertain variable that can be used to compu…

This https://arxiv.org/abs/2404.07315 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Structured Reinforcement Learning for Media Streaming at the Wireless Edge
Media streaming is the dominant application over wireless edge (access) networks. The increasing softwarization of such networks has led to efforts at intelligent control, wherein application-specific actions may be dynamically taken to enhance the user experience. The goal of this work is to develop and demonstrate learning-based policies for optimal decision making to determine which clients to dynamically prioritize in a video streaming setting. We formulate the policy design question as a c…

This https://arxiv.org/abs/2404.07315 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Structured Reinforcement Learning for Media Streaming at the Wireless Edge
Media streaming is the dominant application over wireless edge (access) networks. The increasing softwarization of such networks has led to efforts at intelligent control, wherein application-specific actions may be dynamically taken to enhance the user experience. The goal of this work is to develop and demonstrate learning-based policies for optimal decision making to determine which clients to dynamically prioritize in a video streaming setting. We formulate the policy design question as a c…

Structured Reinforcement Learning for Media Streaming at the Wireless Edge
Archana Bura, Sarat Chandra Bobbili, Shreyas Rameshkumar, Desik Rengarajan, Dileep Kalathil, Srinivas Shakkottai
https://arxiv.org/abs/2404.07315

Structured Reinforcement Learning for Media Streaming at the Wireless Edge
Media streaming is the dominant application over wireless edge (access) networks. The increasing softwarization of such networks has led to efforts at intelligent control, wherein application-specific actions may be dynamically taken to enhance the user experience. The goal of this work is to develop and demonstrate learning-based policies for optimal decision making to determine which clients to dynamically prioritize in a video streaming setting. We formulate the policy design question as a c…