TY - JOUR
T1 - Behavioral model summarisation for other agents under uncertainty
AU - Pan, Yinghui
AU - Ma, Biyang
AU - Tang, Jing
AU - Zeng, Yifeng
N1 - Funding information: Dr. Yinghui Pan conducted this research while she visited the research group in Northumbria University, UK, and would like to thank the supports from NSFC (Grants No.61772442, 61836005 and 61806089).
PY - 2022/1/1
Y1 - 2022/1/1
N2 - Modelling other agents is one of the fundamental research challenges in multiagent decision making under uncertainty. A subject agent expects to optimise its own decisions by correctly modelling what other agents are going to act when they share a common environment and their actions have a joint impact in the transition of environmental states and their individual rewards. However, a true model of the other agent is often unknown due to limited prior knowledge, which particularly occurs to a competitive agent setting. Consequently, the subject agent needs to consider a large number of potential models (the number is infinite in theory) for the other agent, which turns to be computationally hard in its decision making process. A feasible solution is to select a subset of potential models that are representative of all the models for the other agent. In this article, we investigate such a model selection given some known behavioral stereotypes ascribed to the other agents. We propose a new measurement to diversify the selected models under the constraint of the number of models (K models). We formulate the top-K model selection as a sub-modular function optimisation problem and develop a greedy algorithm to obtain high-quality solutions. We develop the proposed model selection techniques in a well-recognised multiagent decision model, namely interactive dynamic influence diagrams, that represents individual agents’ decision making process with the consideration of other agents. By doing this, we can test the performance of the new model selection techniques over two problem domains and provide experimental results in support. Hence, our research in this article contributes to a new model selection technique for modelling other agents therefore improving decision quality of a subject agent interacting with the other agents. It also inspires a creative usage of sub-modular function optimisation in multiagent decision making.
AB - Modelling other agents is one of the fundamental research challenges in multiagent decision making under uncertainty. A subject agent expects to optimise its own decisions by correctly modelling what other agents are going to act when they share a common environment and their actions have a joint impact in the transition of environmental states and their individual rewards. However, a true model of the other agent is often unknown due to limited prior knowledge, which particularly occurs to a competitive agent setting. Consequently, the subject agent needs to consider a large number of potential models (the number is infinite in theory) for the other agent, which turns to be computationally hard in its decision making process. A feasible solution is to select a subset of potential models that are representative of all the models for the other agent. In this article, we investigate such a model selection given some known behavioral stereotypes ascribed to the other agents. We propose a new measurement to diversify the selected models under the constraint of the number of models (K models). We formulate the top-K model selection as a sub-modular function optimisation problem and develop a greedy algorithm to obtain high-quality solutions. We develop the proposed model selection techniques in a well-recognised multiagent decision model, namely interactive dynamic influence diagrams, that represents individual agents’ decision making process with the consideration of other agents. By doing this, we can test the performance of the new model selection techniques over two problem domains and provide experimental results in support. Hence, our research in this article contributes to a new model selection technique for modelling other agents therefore improving decision quality of a subject agent interacting with the other agents. It also inspires a creative usage of sub-modular function optimisation in multiagent decision making.
KW - Behavioral model
KW - Decision model
KW - Intelligent agents
UR - http://www.scopus.com/inward/record.url?scp=85115994561&partnerID=8YFLogxK
U2 - 10.1016/j.ins.2021.09.039
DO - 10.1016/j.ins.2021.09.039
M3 - Article
AN - SCOPUS:85115994561
VL - 582
SP - 495
EP - 508
JO - Information Sciences
JF - Information Sciences
SN - 0020-0255
ER -