Snake Game AI Using Deep Neural Networks

MEMBER 1 MEMBER 2(TEAM LEADER) MEMBER 3:

KETHACHAITANYA S.LALITH KUMAR P.VARUN
REGNO:22BCE9090 REG NO: 22BCE9938 REGNO:22BCE7303
SLOT:A2 SLOT: A2 SLOT:A2

Abstract— high computational effort, which might be wasteful for a

comparably straightforward game environment.
The evolution of artificial intelligence (AI) in game-playing To overcome these issues, this study investigates a different
has led to significant advancements in decision-making and solution: employing a Deep Neural Network (DNN) learned
autonomous control. This project explores a Deep Neural Network through supervised learning to play the Snake game
(DNN) approach to playing the classic Snake game using
independently. Rather than learning through trial and error
supervised learning, as an alternative to reinforcement learning
techniques like Deep Q-Networks (DQN). The proposed method
as in reinforcement learning, the new method is to train a
follows five stages: (1) implementing the Snake game using DNN on pre-obtained gameplay data created by a rule-based
pygame, (2) collecting gameplay data from a rule-based agent, (3) bot. The main benefits of this method are:
training a DNN model to predict optimal moves based on game Computational Efficiency: Unlike reinforcement learning,
states, (4) deploying the trained model to play the game, and (5) which requires continuous exploration and policy updates,
evaluating its performance.The training dataset is generated using a supervised learning allows the model to directly learn from
heuristic-based bot, and a fully connected feedforward neural labeled data, reducing training time.
network is trained using cross-entropy loss and the Adam Stability and Generalization: The learned model picks up
optimizer in PyTorch. The trained model generalizes well to
regular decision-making habits from organized data,
unseen game scenarios, demonstrating intelligent movement
decisions that improve gameplay performance. Results indicate resulting in more stable and predictable performance.
that a supervised learning-based DNN can effectively replace Simplified Training Process: DNN training with labeled
reinforcement learning methods, offering a computationally examples avoids the requirement of intricate reward
efficient and stable solution.This work highlights the feasibility of engineering or hyperparameter tuning, which are crucial in
DNNs for autonomous gameplay and opens avenues for further reinforcement learning.
enhancements, including convolutional networks for visual inputs
and hybrid approaches that integrate supervised and reinforcement
learning for adaptive decision-making. II. LITERACTURE SURVEY

A. Overview of the Snake Game in AI Research

I. INTRODUCTION (HEADING 1)
Artificial Intelligence (AI) has improved tremendously over The Snake game is a widely used benchmark in AI research
the past few years, especially in the field of game-playing due to its simple rules yet challenging dynamics. The
AI, with deep learning methods ushering in revolutionary game's objective is for the snake to navigate the grid, collect
breakthroughs. AI systems have been able to beat human food, and avoid collisions, making it a dynamic sequential
players in traditional board games like Chess, Go, and decision-making task.
Shogi, as well as intricate computer games like Dota 2 and
Various AI techniques have been applied to the Snake game,
StarCraft II. Typically, game-playing AI uses
including:
Reinforcement Learning (RL) methods, whereby an agent
learns by interacting with the environment and adapting a
B. Rule-Based Approaches
reward-based policy. But reinforcement learning techniques,
like Deep Q-Networks (DQN), require large amounts of Rule-based techniques were the earliest AI methods
exploration, computation, and training time and are applied to the Snake game. They employ pre-established
therefore unsuitable for easier tasks.The Snake game, an logic to decide based on the state of the game. Some
arcade game, is an intriguing problem for AI because it has common techniques are:
a dynamic environment and decision-making constraints. • Greedy Algorithms: Head towards the food in the
The game asks for an agent to move in a Bounded grid, direction of the shortest Euclidean distance.
gather food, and evade crashes while accumulating its score.
Most current AI solutions for the Snake game utilize Q- • Hamiltonian Cycles: Specialized paths that ensure
the snake never gets trapped (Salvador et al., 2009)
learning or DQN-based reinforcement learning to deliver
[1].
optimal performance. Although these work well, they tend
to use millions of iterations, hyperparameter tuning, and • A* Pathfinding Algorithm: Determines the shortest
path to the food source without hitting obstacles.
Although rule-based approaches guarantee determinable
behavior, they do not support adaptability and generalization
in complicated situations.
 C. Reinforcement Learning Approaches Nevertheless, the majority of current DNN-based methods
Reinforcement Learning (RL) has been extensively used in are based on reinforcement learning, with few attempts to
the Snake game, enabling the agent to acquire policies by explore supervised learning techniques. This work seeks to
trial-and-error interaction with the environment. The most bridge this gap by proving the efficacy of supervised
popular RL methods are: learning-based DNNs for Snake game AI.
• Q-Learning: A tabular approach where the agent F. Comparative Analysis: Supervised Learning vs.
learns state-action values (Q-values) by exploration Reinforcement Learning
(Watkins, 1989) [2].
• Deep Q-Networks (DQN): Proposed by Mnih et al. Feature Supervised Learning Reinforcement Learning
(2015) [3], DQN merges Q-learning with deep (DNN) (DQN)
Slow (Millions of
neural networks to estimate Q-values in high- Training Time Fast iterations)
dimensional state spaces. Computationa
• Double DQN (DDQN): Van Hasselt et al. (2016) l Cost
Low High
[4] introduced DDQN to overcome the Stability Unstable
Stable
overestimation bias of DQN.
While successful, RL-based approaches have a number of Generalizatio Limited (depends on
High
n dataset)
drawbacks:
Data Pre-collected labeled
• High computational cost Requirement data
Self-generated
• Long training durations This comparison highlights that supervised learning offers
• Instability of training faster training and greater stability, making it a suitable
• Hyperparameter sensitivity choice for games like Snake with deterministic rules.

D.Supervised Learning in Game AI

Supervised Learning (SL) has been investigated as a III. METHODOLOGY
substitute for reinforcement learning for game AI in recent Before you begin to format your paper, first write and
research. Supervised learning differs from RL in that it save the content as a separate text file. Complete all content
trains the model with pre-existing labeled data, and hence it and organizational editing before formatting. Please note
is more efficient and stable. Supervised learning has been sections A-D below for more information on proofreading,
utilized in numerous game-playing tasks, such as: spelling and grammar.
• Chess Move Prediction (Silver et al., 2018) [5]:
Supervised learning was employed to pre-train Keep your text and graphic files separate until after the
text has been formatted and styled. Do not use hard tabs, and
AlphaZero's policy network prior to reinforcement
limit use of hard returns to only one return at the end of a
learning.
paragraph. Do not add any kind of pagination anywhere in
• Atari Games (Hussein et al., 2017) [6]: Showed the paper. Do not number text heads-the template will do that
that imitation learning using supervised data was for you.
able to obtain competitive performance on a
number of Atari games. A. Game Environment
• Pac-Man (Sharma et al., 2020) [7]: An expert-
The Snake game environment is implemented using the
trained DNN played Pac-Man successfully without
Pygame library, which provides a structured framework for
reinforcement learning.
game development. The game consists of a grid where the
These works imply that supervised learning is most
snake moves to collect food while avoiding collisions with
appropriate for well-structured, rule-based games such as
itself and the boundaries. The game state is represented by a
Snake where labeled data is easily obtainable.
structured array containing:
• The position of the snake's head.
E. Deep Neural Networks for Game AI • The positions of the snake's body segments.
Deep Neural Networks (DNNs) have been extensively • The location of the food.
utilized in game-playing AI because they can approximate • The boundaries of the game grid.
complicated functions. DNNs have been used in both A fixed time step controls the movement of the snake, and a
supervised and reinforcement learning. The architecture is scoring system is implemented based on the number of food
usually composed of fully connected layers or convolutional items consumed.
layers based on the input representation.
In the case of the Snake game, feedforward networks with The Snake game is programmed with the Pygame library,
full connectivity are usually employed to translate game with a grid-based environment for the AI agent to move
states into best actions. The most significant strengths of around in, collect food, and crash into obstacles.
DNNs for game AI are:
B. Data Collection
• High expressiveness
• Generalization ability To train the DNN using supervised learning, a dataset of
• Effective training on labeled data game states and corresponding optimal moves is required.
This is achieved through:
 • Heuristic-Based Agent: A rule-based agent is
designed using simple heuristics, such as moving
toward the food and avoiding collisions.
• Game State Representation: Each state is
represented as a feature vector, encoding
information about the relative position of the food,
walls, and the snake's body.
• Action Labels: The best move (up, down, left,
right) is assigned based on predefined rules
ensuring optimal performance.
• Dataset Generation: Thousands of game state-
action pairs are recorded by running the heuristic
agent in different game scenarios. E. Deployment and Testing
A heuristic rule-based agent is created to produce labeled Once the model is trained, it is deployed to play the Snake
training data. The game situations (positions on the grid of game autonomously. The AI agent selects moves by feeding
the snake, food, and obstacles) are captured and the best the current game state into the trained DNN, choosing the
moves using pre-defined strategies. action with the highest probability output. Performance is
evaluated based on:
C. Model Architecture • Average Score: The number of food items
A fully connected Deep Neural Network (DNN) is designed collected per game.
to predict optimal moves given a game state. The • Survival Time: The duration for which the AI
architecture includes: avoids collisions.
• Input Layer: Accepts the game state representation • Comparison with Baseline: The trained model is
as a vector. compared against a random agent and the heuristic
• Hidden Layers: Multiple fully connected layers rule-based agent to quantify improvement.
with ReLU activation functions to learn complex
decision-making patterns. IV. MATHEATICAL FORMULATION
• Output Layer: A softmax layer that outputs
A. Game State Representation
probabilities for the four possible moves (up,
down, left, right). Let the game state at time step be represented as a feature
This model enables the AI to infer the best action based on vector:
learned patterns from the training data. St=[Xhead,Yhead,Xfood,Yfood,Dwall,Dbody]
where:
• Xhead,Yhead,are the coordinates of the snake's head.
D.Training Process • Xfood,Yfood .are the coordinates of the food.
The training process involves: • Dwall is the distance to the nearest wall.
• Preprocessing: Normalizing input features and • Dbody is the distance to the nearest body segment.
balancing the dataset.
• Loss Function: Cross-entropy loss is used for
multi-class classification. B. Action Prediction Model
• Optimizer: The Adam optimizer is employed for The Deep Neural Network (DNN) models the probability
efficient gradient descent and weight updates. distribution of actions given the state:
• Training Framework: The model is implemented in
PyTorch, and trained using mini-batch gradient P(A∣St)=fθ(St)
descent.
where fθf_\thetafθ represents the neural network with
• Evaluation Metrics: Training accuracy, loss curves,
parameters θ\thetaθ.
and validation performance are monitored to
The AI agent selects an action AtA_tAt based on the highest
prevent overfitting and ensure generalization.
probability output:
The dataset is split into training and validation sets, ensuring
that the model generalizes well to unseen game scenarios. At=arga∈{up,down,left,right}maxP(A=a∣St)
C. Loss Function
Since the model is trained using supervised learning, we use
the cross-entropy loss function for multi-class
classification:

L(θ)=−i=1∑Nyilog(yi^)
where:
• yi is the true action label (optimal move from
heuristic agent).
 • yi^ is the predicted probability for the correct CONCLUSION
action. This work proposes a Deep Neural Network (DNN)-based
supervised learning method for creating an AI agent to play
the Snake game. Contrary to the conventional
D.Optimization Reinforcement Learning (RL)-based techniques that demand
long training times and computational power, our method
The model is trained using gradient descent with the Adam
exploits supervised learning through training the model
optimizer:
using gameplay data from a heuristic agent. The model
θ=θ−α∇θL(θ)
learns well to predict best actions given game states, and it
α is the learning rate. This provides effective and stable
exhibits stable and efficient performance.
weight updates during training.
Experimental results show that the AI agent based on DNN
This mathematical expression specifies how the AI sees the
generalizes to new game situations well, displaying smart
game state, makes optimal action predictions, and learns
movement behaviors, longer lifetime, and efficient food
from labeled data to enhance gameplay performance.
collection compared to naive or random rule-based agents.
Additionally, the use of cross-entropy loss function and
Adam optimizer ensures speedy convergence and model
E. EXPERIMENTAL RESULTS stability. Supervised learning's computational efficiency
makes it a reasonable alternative to deep reinforcement
learning for structured game environments such as Snake.
Still, the method has some drawbacks. As the model is
trained from labeled data derived from a fixed heuristic
agent, the quality of training data becomes the limiting
factor in its ability to make decisions. In highly dynamic and
sophisticated game situations, reinforcement learning or
combined models would improve adaptability and
performance further.

REFERENCES
[1] [1] A. Sebastianelli, M. Tipaldi, S. L. Ullo, and L. Glielmo, "A Deep
Q-Learning based approach applied to the Snake game," arXiv
preprint arXiv:2301.11977, 2023. researchgate.net
[2] [2] J. D. Abraham, "Deep Reinforcement Learning for Snake: How
well does a deep Q-network play the Snake game?" M.Sc. thesis,
Eindhoven University of Technology, Netherlands, 2021.
research.tue.nl
[3] [3] S. Liberata Ullo, M. Tipaldi, A. Sebastianelli, and L. Glielmo, "A
Deep Q-Learning based approach applied to the Snake game," CEAS
Electronic Journal, vol. 13, no. 1, pp. 10-20, 2021.
researchgate.net+1CEAS Journal+1
[4] [4] N. Zhou, "Teaching an AI to Play the Snake Game Using
Reinforcement Learning," Medium, 2021. Medium+1Reddit+1
[5] [5] A. Sebastianelli, M. Tipaldi, S. L. Ullo, and L. Glielmo, "Training
an AI agent to play a Snake Game via Deep Reinforcement
Learning," International Journal of Scientific Research in Science
and Technology, vol. 7, no. 1, pp. 81-89, 2020.
researchgate.net+1ijsrst.com+1
[6] [6] S. L. Ullo, M. Tipaldi, A. Sebastianelli, and L. Glielmo, "A
Memory Efficient Deep Reinforcement Learning Approach For Snake
Game," arXiv preprint arXiv:2301.11977, 2023. arxiv.org
[7] [7] A. K. Singh and S. K. Singh, "Snake Game: A genetic neural
network approach," International Journal of Computer Applications,
vol. 6, no. 1, pp. 45-50, 2022. CloudFront
[8] [8] J. Andersson and J. Johansson, "Deep Reinforcement Learning for
Snake," M.Sc. thesis, Uppsala University, Sweden, 2019.