Generative Methods for Solving Interactive Dynamic Influence Diagrams

Interactive Dynamic Influence Diagrams (I-DIDs) face a long-term standing challenge of dealing with unknown behavior of other agents who share a common environment with a subject agent in a partially observable stochastic game. A diversity-based technique provides promising solutions in a data-driven approach to I-DIDs; however, it triggers the new challenge of limited historical data of agents’ interactions for generating potential behaviors of other agents. To tackle this issue, we propose a Generative Interactive Dynamic Influence Diagram (GI-DID) framework, which integrates various generative methods and incorporates a novel perplexity-based metric to enhance policy tree ensemble diversity. Within this framework, we propose the Wasserstein Divergence Adversarial Autoencoder (WDAE) for behavior generation, which diversifies and generates potential behaviors, improving modeling and prediction capabilities. Experimental results on two classic problem domains demonstrate the WDAE-based method’s effectiveness in enhancing I-DID performance, contributing to advancements in opponent modelling and paving the way for future explorations of multiagent decision making in an open artificial intelligent (AI) environment.