UR-78 Transforming Game Play: A Comparative Study of CNN and Transformer based Q-Networks in Reinforcement Learning

UR-78 Transforming Game Play: A Comparative Study of CNN and Transformer based Q-Networks in Reinforcement Learning

https://www.kennesaw.edu/ccse/events/computing-showcase/sp24-cday-program.php

In this study we investigate the performance of Deep Q-Networks utilizing Convolutional Neural Networks (CNNs) and Transformer architectures across 3 different Atari Games. The advent of DQNs have significantly advanced Reinforcement Learning, enabling agents to directly learn optimal policy from high dimensional sensory inputs from pixel or RAM data. While CNN based DQNs have been extensively studied and deployed in various domains Transformer based DQNs are relatively unexplored. Our research aims to fill this gap by benchmarking the performance of both DCQNs and DTQNs across the Atari games' Asteroids, Space Invaders and Centipede. Our research finds that our Transformer Agent learned slower than the CNN-based agent, and was slower to learn game-extending policies.

https://digitalcommons.kennesaw.edu/cday/Spring_2024/Undergraduate_Research/7