Research on Autonomous Decision-Making Agents for Super Mario Bros Based on Various Deep DQN Architectures

📅 2022 🔖 Course project

This work is the final project for the Advanced ML course at the University of Lincoln. This project implements and compares five DQN-based agents with different network architectures for autonomous gameplay in the Super Mario Bros environment. The implemented agents include Nature DQN (CNN), Nature DQN (RNN), Nature DQN (Transformer), Double DQN (From the PyTorch official tutorial), and Dueling DQN. All models are value-based deep reinforcement learning methods grounded in Q-learning, approximating the action-value function \( Q(s, a) \) using deep neural networks. During training, the agents leverage experience replay (Replay Buffer), ε-greedy exploration, and target networks to improve training stability and sample efficiency. Under a constrained training budget of 3000 episodes, Dueling DQN achieved the highest average reward, while the Transformer-based architecture outperformed traditional CNNs in terms of average reward, highlighting its potential for future DRL research. These findings suggest that integrating Dueling structures with Transformer-based networks may lead to more effective deep reinforcement learning agents. More details about this work, please check this report. The code has been released on github.

Acknowledgements

This work was supported by the Engineering and Physical Sciences Research Council and AgriFoRwArdS CDT [EP/S023917/1]. Thanks to the support of the University of Lincoln.