Prof. Tamer Basar, University of Illinois Urbana-Champaign

Title: RL for Multi-Agent Dynamical Systems with Asymmetric Information

Abstract: Decision making in dynamic uncertain environments with multiple agents having possibly misaligned objectives arises in many disciplines and application domains, including control (particularly networked control, such as control and operation of multiple robots, unmanned vehicles, mobile sensor networks, and the smart grid), communications (particularly in transmission of information to multiple destinations under privacy constraints), distributed optimization (particularly with topological and informational constraints), social networks (such as problems of consensus and dissensus), and economics (such as inducement of behavior via incentive policies). A natural framework, and a comprehensive one, for modeling, optimization, and analysis in such systems is the one provided by stochastic dynamic games, which accommodates different solution concepts depending on how the interactions among the agents are modeled. The inherent asymmetry in information across the agents, with them not operating under the same (and consistent) modeling assumptions, and with strategic interactions taking place in neighborhoods and propagating across the network create major challenges in the decision-making process, necessitating each agent to operate in a non-stationary environment and develop beliefs on others, with the belief generation process leading to what is known as second-guessing phenomenon. Another challenge presents itself in scalability of the decision process, as the size of the population of the agents grows. This latter challenge actually turns out to be a blessing in itself, under some (realistic) structural specifications, as in the high population setting when the agents become infinitesimal entities, making the underlying dynamic game asymptotically belonging to the class of mean field games (MFGs), a topic that has attracted intense research activity in recent years.

This workshop talk will provide an overview of recent developments in the landscape described above, focusing on some foundational results for both model-based and model-free settings, with the latter involving data-driven policy design, requiring reinforcement learning, zero-order stochastic optimization, and finite-sample analysis. Both single and multiple population scenarios will be covered. Discussion of selected applications and future challenges will conclude the talk.