Tallon AI: Probabilistic Decision-Making in Mean Arena
📅 2022 🔖 Course project
This work is the final project for the Advanced AI course at the University of Lincoln. I designed an AI decision-making agent based on probabilistic reasoning and optimal policy planning, integrating Markov Decision Processes, value iteration, belief state updates and others to enable rational sequential decision-making under both fully observable and partially observable environments. Under fully observable conditions, the game is modeled as a Markov Decision Process (MDP), in which a state transition model and a reward function are constructed, and the optimal policy is obtained using value iteration. Within this framework of maximizing expected long-term reward, Tallon is encouraged to avoid enemies and pits while prioritizing the collection of bonuses. Under partially observable conditions, the problem is further extended to a Partially Observable Markov Decision Process (POMDP), where belief states over enemy locations are maintained and combined with a prediction mechanism to approximate the incomplete information and map it back to an MDP for decision-making. In addition, reward shaping strategies based on distance and aggregated risk are introduced to enhance survivability and overall performance. Experimental results demonstrate that the proposed approach exhibits good robustness and adaptability across different map sizes and parameter configurations. More details about this work, please check this report. The code has been released on github.
Acknowledgements
This work was supported by the Engineering and Physical Sciences Research Council and AgriFoRwArdS CDT [EP/S023917/1]. Thanks to the support of the University of Lincoln.