Deep learning can be considered as a subset of machine learning.

It is a field
that is based on learning and improving on its own by examining computer
algorithms. While machine learning uses simpler concepts, deep learning
works with artificial neural networks, which are designed to imitate how
humans think and learn. Until recently, neural networks were limited by
computing power and thus were limited in complexity. However,
advancements in Big Data analytics have permitted larger, sophisticated neural
networks, allowing computers to observe, learn, and react to complex
situations faster than humans. Deep learning has aided image classification,
language translation, speech recognition. It can be used to solve any pattern
recognition problem and without human intervention.

Artificial neural networks, comprising many layers, drive deep learning. Deep
Neural Networks (DNNs) are such types of networks where each layer can
perform complex operations such as representation and abstraction that make
sense of images, sound, and text. Considered the fastest-growing field in
machine learning, deep learning represents a truly disruptive digital
technology, and it is being used by increasingly more companies to create new
business models.

Neural networks are layers of nodes, much like the human brain is made up of
neurons. Nodes within individual layers are connected to adjacent layers. The
network is said to be deeper based on the number of layers it has. A single
neuron in the human brain receives thousands of signals from other neurons.
In an artificial neural network, signals travel between nodes and assign
corresponding weights. A heavier weighted node will exert more effect on the
next layer of nodes. The final layer compiles the weighted inputs to produce an
output. Deep learning systems require powerful hardware because they have a
large amount of data being processed and involves several complex
mathematical calculations. Even with such advanced hardware, however,
training a neural network can take weeks.
To understand how an artificial neuron works, we should first understand
how a biological neuron works.

Dendrites

These receive information or signals from other neurons that get

connected to it.

Cell Body

Information processing happens in a cell body. These take in all the

information coming from the different dendrites and process that
information.

Axon

It sends the output signal to another neuron for the flow of information.
Here, each of the flanges connects to the dendrite or the hairs on the next
one.

The image shown below depicts an ANN.

The network starts with an input layer that receives input in data form.

The lines connected to the hidden layers are called weights, and they add
up on the hidden layers. Each dot in the hidden layer processes the
inputs, and it puts an output into the next hidden layer and, lastly, into
the output layer.

Looking at the above two images, you can observe how an ANN replicates
a biological neuron.

 Input to a neuron - input layer

 Neuron - hidden layer

 Output to the next neuron - output layer

A neural network is a system of hardware or software patterned after the

operation of neurons in the human brain. Neural networks, also called
artificial neural networks, are a means of achieving deep learning.

There are five recognized types of neural networks.

 Single-layer feed-forward network.

  Multilayer feed-forward network.

 Single node with its own feedback.

 Single-layer recurrent network.

 Multilayer recurrent network.

Different types of Neural Networks in Deep Learning

 Artificial Neural Networks (ANN)

 Convolution Neural Networks (CNN)
 Recurrent Neural Networks (RNN)
ANN
A single perceptron (or neuron) can be imagined as a Logistic
Regression. Artificial Neural Network, or ANN, is a group of
multiple perceptrons/ neurons at each layer. ANN is also known as
a Feed-Forward Neural network because inputs are processed
only in the forward direction:

ANN

As you can see here, ANN consists of 3 layers – Input, Hidden and
Output. The input layer accepts the inputs, the hidden layer
processes the inputs, and the output layer produces the result.
Essentially, each layer tries to learn certain weights.

ANN can be used to solve problems related to:

 Tabular data
 Image data
 Text data

Advantages of Artificial Neural Network (ANN)

Artificial Neural Network is capable of learning any nonlinear
function. Hence, these networks are popularly known
as Universal Function Approximators. ANNs have the capacity
to learn weights that map any input to the output.

One of the main reasons behind universal approximation is

the activation function. Activation functions introduce nonlinear
properties to the network. This helps the network learn any
complex relationship between input and output.

Perceptron

As you can see here, the output at each neuron is the activation
of a weighted sum of inputs. But wait – what happens if there is
no activation function? The network only learns the linear function
and can never learn complex relationships. That’s why:

An activation function is a powerhouse of ANN!

RNN
RNN works on the principle of saving the output of a particular layer and
feeding this back to the input in order to predict the output of the layer. It
is the first algorithm that remembers its input, due to an internal memory,
which makes it perfectly suited for machine learning problems that
involve sequential data. It is one of the algorithms behind the scenes of
the amazing achievements seen in deep learning over the past few years.

Below is how you can convert a Feed-Forward Neural Network into a

Recurrent Neural Network:

The nodes in different layers of the neural network are compressed to

form a single layer of recurrent neural networks. A, B, and C are the
parameters of the network.

In a RNN the information cycles through a loop. When it makes a

decision, it considers the current input and also what it has
learned from the inputs it received previously.
A usual RNN has a short-term memory. In combination with a
LSTM they also have a long-term memory .

CNN
A convolutional neural network, or CNN, is a deep learning
neural network designed for processing structured arrays of data
such as images. Convolutional neural networks are widely used
in computer vision and have become the state of the art for many
visual applications such as image classification, and have also
found success in natural language processing for text
classification.

Convolutional neural networks are very good at picking up on

patterns in the input image, such as lines, gradients, circles, or
even eyes and faces. It is this property that makes convolutional
neural networks so powerful for computer vision. Unlike earlier
computer vision algorithms, convolutional neural networks can
operate directly on a raw image and do not need any
preprocessing.

A convolutional neural network is a feed-forward neural network,

often with up to 20 or 30 layers. The power of a convolutional
neural network comes from a special kind of layer called the
convolutional layer.

Convolutional neural networks contain many convolutional layers

stacked on top of each other, each one capable of recognizing
more sophisticated shapes. With three or four convolutional
layers it is possible to recognize handwritten digits and with 25
layers it is possible to distinguish human faces.

The usage of convolutional layers in a convolutional neural

network mirrors the structure of the human visual cortex, where
a series of layers process an incoming image and identify
progressively more complex features.