Genetic Algorithm

Dr. Zoya Khalid

[email protected]
Evolutionary Algorithm (EA)
• An evolutionary algorithm is a type of optimization algorithm inspired by the
process of natural selection. It is commonly used to solve complex optimization
problems where traditional methods may be inefficient.

• The algorithm mimics the process of evolution by maintaining a population of

candidate solutions. These solutions are then subjected to genetic operators such
as mutation and crossover to generate new candidate solutions. The fitness of
each solution is evaluated based on how well it performs with respect to the
optimization problem objective.
Genetic Algorithm as an example of
EA
Genotype - Phenotype
•Genotype − Genotype is the population in the computation space. In the computation space, the solutions
are represented in a way which can be easily understood and manipulated using a computing system.
•Phenotype − Phenotype is the population in the actual real world solution space in which solutions are
represented in a way they are represented in real world situations.
 Intuition behind Genetic Algorithms
• It is not the strongest of the species that survives, nor the most intelligent , but the one most
responsive to change.

• Let’s take a hypothetical situation where, you are head of a country, and in order to keep your city

safe from bad things, you implement a policy like this. • You select all the good people, and ask them

to extend their generation by having their children. • This repeats for a few generations.

• You will notice that now you have an entire population of good people.

• So the basic idea was that we changed the input (i.e. population) such that we get better output
(i.e. better
country).
Biological Inspiration

• Cells are the basic building block of all living things.

• Therefore in each cell, there is the same set of chromosomes. Chromosome are
basically the strings of DNA.
 Biological Inspiration
• Traditionally, these chromosomes are represented in binary as strings
of 0’s and 1’s.
A chromosome consists of genes, commonly referred as blocks of DNA, where each gene encodes a
specific trait, for example hair color or eye color.

What is a Genetic Algorithm?

• Let’s get back to the example we discussed above and summarize what we

did. • Firstly, we defined our initial population as our countrymen.

• We defined a function to classify whether is a person is good or bad. •

Then we selected good people for mating to produce their off-springs.

• And finally, these off-springs replace the bad people from the population and this
process repeats.

• This is how genetic algorithm actually works, which basically tries to mimic the
human evolution to some extent.
What is a Genetic Algorithm?
• So to formalize a definition of a genetic algorithm, we can say that it is an optimization technique,
which tries to find out such values of input so that we get the best output values or results.

• The working of a genetic algorithm is also derived from biology, which is as shown in the image
below.

Steps Involved in Genetic Algorithm

• Let’s say, you are going to spend a month in the wilderness. Only thing you are
carrying is the backpack which can hold a maximum weight of 30 kg. Now you
 have different survival items, each having its own “Survival Points” (which are
given for each item in the table). So, your objective is maximize the survival
points.
• Here is the table giving details about each item.

Initialization
• To solve this problem using genetic algorithm, our first step would be defining our population. So
our population will contain individuals, each having their own set of chromosomes.
• We know that, chromosomes are binary strings, where for this problem 1 would mean that the
following item is taken and 0 meaning that it is dropped.

This set of chromosome is considered as our initial population.

Fitness Function
• Let us calculate fitness points for our first two
chromosomes. • For A1 chromosome [100110],

Fitness Function
• Similarly for A2 chromosome [001110],

So, for this problem, our chromosome will be considered as more fit when it contains more survival
points.
Therefore chromosome 1 is more fit than chromosome 2.

Fitness Function
• There are two main categories of methods for using fitness in order to support selection
1. Deterministic methods
• Fitness function is evaluated using fixed set of criteria
• This approach ensures that each solution is evaluated consistently and objectively. • Involve
for example the selection of n most fit individuals in a population for recombination.
• They are generally discredited because they tend to develop a population that reaches the local
maximum and stops evolving

2. Stochastic Method
• In the stochastic method, the fitness function incorporates some element of randomness or
probabilistic evaluation.
• This approach allows for more flexibility in evaluating solutions and can capture a wider
range of possible solutions.

Roulette Wheel Selection as Stochastic

 Method
Roulette wheel selection is a popular method used in genetic algorithms to select individuals for
reproduction based on their fitness. It is considered one of the most suitable methods due to several

reasons:

1. Fitness-proportional selection: Roulette wheel selection assigns a probability of selection to

each individual in the population based on their fitness value The higher the fitness value,
the higher the probability of being selected. This ensures that individuals with higher
fitness have a better chance of being selected for reproduction, leading to the survival of
the fittest.

Roulette Wheel Selection

2. Exploration and exploitation: The roulette wheel selection method strikes a balance between
exploration and exploitation. By assigning probabilities based on fitness, it allows for exploration by
giving lower fitness individuals a chance to be selected. At the same time, it also exploits the higher
fitness individuals by giving them a higher probability of selection. This helps in maintaining
diversity in the population while also converging towards optimal solutions.

3. Simplicity and efficiency: Roulette wheel selection is a straightforward and easy-to-implement

method. It does not require complex computations or parameter tuning. The selection process can
be efficiently performed by generating a random number and traversing the population to find the
selected individual based on the calculated probabilities. This simplicity and efficiency make it a
popular choice for genetic algorithm implementations.

4. Reproduction proportionality: Since the selection probabilities are directly proportional to the

fitness values, the roulette wheel selection method ensures that the reproductive contribution of
each individual is proportional to its fitness. This helps in maintaining the genetic
Selection
• Now, we can select fit chromosomes from our population which can mate and create their off
springs.
• General thought is that we should select the fit chromosomes and allow them to produce off
springs. But that would lead to chromosomes that are more close to one another in a few next
generation, and therefore less diversity.
• Therefore, we generally use Roulette Wheel Selection method.
Roulette Wheel Selection
• Consider a wheel, and let’s divide that into m divisions, where m is the number of
chromosomes in our populations. The area occupied by each chromosome will be
proportional to its fitness value.

Roulette Wheel Selection

So, now this wheel is rotated and the region of wheel which comes in
front of the fixed point is chosen as the parent. For the second parent,
the same process is repeated.
Crossover
• So in this previous step, we have selected parent chromosomes that will produce
off-springs. So in biological terms, crossover is nothing but reproduction.
• So let us find the crossover of chromosome 1 and 4, which were selected in the
previous step.

Crossover
 If you take two

crossover point, then it will called as multi point crossover which is as shown below.

Mutation
• Now if you think in the biological sense, are the children produced have the same
traits as their parents? The answer is NO.
• During their growth, there is some change in the genes of children which makes
them different from its parents.
• This process is known as mutation, which may be defined as a random tweak in
the chromosome, which also promotes the idea of diversity in the population.

Evaluation of Fitness Function

• The off-springs thus produced are again validated using our fitness function, and
if considered fit then will replace the less fit chromosomes from the population.
• But the question is how we will get to know that we have reached our best
possible solution?

• So basically there are different termination conditions, which are listed below:
• There is no improvement in the population for over x iterations.
• We have already predefined an absolute number of generation for our algorithm.
• When our fitness function has reached a predefined value.

Example
• Imagine a population of birds trying to find the best way to build their nests. The
birds have different characteristics, such as beak size, wing shape, and feather
color, which can affect their ability to build a successful nest.
• The goal is to find the best combination of these characteristics that will result in
the most effective nest-building behavior.

In the initial population, each bird's characteristics are represented as a set of

genes or traits. These genes encode the information needed for nest-building,
such as beak length, wing length, and feather color.

The first step in a genetic algorithm is the evaluation of the fitness of each
individual in the population. In our example, the fitness of a bird can be
determined by how well it can build a nest. Birds with better nest-building skills
will have a higher fitness score.
Example
• The next step is the selection process, where individuals with higher fitness
scores have a higher chance of reproducing. This mimics natural selection, where
individuals with advantageous traits are more likely to survive and pass on their
genes to the next generation.
•

In genetic algorithms, reproduction is simulated through genetic operators such

as crossover and mutation. Crossover involves combining genes from two parents
to create offspring with a combination of their traits. Mutation introduces
random changes in the genes, sim
Application in Sequence Alignment
• Sequence alignment is a fundamental problem in bioinformatics, where the goal
 is to find the best matching alignment between two or more biological
sequences. Genetic algorithms can be used to solve this problem by iteratively
improving candidate alignments through a process of evolution.
•
1. Initialization:
• Start by generating a population of candidate alignments. Each candidate
alignment represents a potential solution, and is typically represented as a
string of characters where gaps (indicated by "-") are inserted to align the
sequences.
Application in Sequence Alignment
2. Fitness evaluation:
Assign a fitness score to each candidate alignment based on how well it
matches the desired criteria. In the case of sequence alignment, the fitness
 score can be determined by evaluating the similarity or dissimilarity
between the aligned sequences. Common scoring methods include the
Needleman-Wunsch or Smith-Waterman algorithms.
3. Selection:
Select a subset of candidate alignments from the population to serve as
parents for the next generation. The selection process is typically based on
the fitness scores of the alignments, where alignments with higher fitness
scores have a higher probability of being selected
Application in Sequence Alignment
• 4. Crossover:
Generate new candidate alignments by combining the selected
alignments through crossover. Crossover involves exchanging
segments of the parent alignments to create new alignments. This process
 mimics the genetic recombination that occurs in natural evolution.

5. Mutation:
Introduce random changes to the newly generated alignments to maintain
genetic diversity in the population. These changes can include insertions,
deletions,
Application of Genetic Algorithm in Feature
Selection
• 1. Classification: Genetic algorithms can be used to select a subset of features
that are most relevant for a particular classification task. By evaluating the fitness
of different feature subsets based on their classification performance, genetic
algorithms can efficiently search for an optimal subset of features.
2. Regression: Genetic algorithms can also be applied to feature selection in
regression tasks. By considering the relationship between the selected features
and the target variable, genetic algorithms can identify the most informative
features for accurate regression modeling.

3. Bioinformatics: In bioinformatics, genetic algorithms are commonly used to

select relevant features from high-dimensional gene expression data. By
identifying a subset of genes that are most informative for disease classification,
genetic algorithms can aid in the discovery of biomarkers and improve diagnostic
accuracy
Application of Genetic Algorithm in Feature
Selection
• 4. Image recognition: Genetic algorithms can be applied to feature selection in
image recognition tasks. By selecting a subset of features that capture important
visual patterns, genetic algorithms can improve the efficiency and accuracy of
image classification algorithms.

5. Text classification: Genetic algorithms can be used to select relevant features

from a large set of textual features for text classification tasks. This can help in
reducing the dimensionality of the feature space and improving the performance
of text classification models.
Application of Genetic Algorithm
Feature Selection
• How do you select features that are important in prediction of the target variable? You
always look at the feature importance of some model, and then manually decide the
threshold, and select the features which have importance above that threshold.

• Is there any better way to deal with this kind of situations? Actually one of the most
advanced algorithms for feature selection is genetic algorithm.

• The method here is completely same as the one we did with the knapsack problem.

• We will again start with the population of chromosome, where each chromosome will be
binary string. 1 will denote “inclusion” of feature in model and 0 will denote “exclusion”
of feature in the model.

Feature Selection
• And another difference would be that the fitness function would be
changed. The fitness function here will be our accuracy metric of the
competition. The more accurate our set of chromosome in predicting
value, the more fit it will be.
Advantages of GAs
 Examples not suitable for Genetic Algorithm
1. Continuous and smooth search space:
It is suitable for the problems in discrete space where there are distinct solutions.
2. Problems with small number of variables:
GA are more effective when dealing with problems that have large number of
variables.

3. Time-Sensitive problems:
GAs are computationally expensive and may require a significant amount of time to
converge to an optimal solution.
References
1. Biologically Inspired Optimization Methods: An Introduction by M.
Wahde ,Chalmers University of Technology, Sweden