Scribd
Upload
114 views

Unit4 - Chain Rule and Backpropagation

Uploaded by

jini shine
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
114 views

Unit4 - Chain Rule and Backpropagation

Uploaded by

jini shine
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 4

CHAIN RULE AND BACKPROPAGATION

In deep learning, the chain rule is a fundamental concept used in


backpropagation, an algorithm for efficiently computing gradients during the
training of neural networks.

Backpropagation is an algorithm that backpropagates the errors from the


output nodes to the input nodes. Therefore, it is simply referred to as the
backward propagation of errors. It uses in the vast applications of neural
networks in data mining like Character recognition, Signature verification,
etc.
Backpropagation is a widely used algorithm for training feedforward neural
networks. It computes the gradient of the loss function with respect to the
network weights. It is very efficient,

Neural Network:

Neural networks are an information processing paradigm inspired by the


human nervous system.

Features of Backpropagation:

1. it is the gradient descent method as used in the case of simple perceptron


network with the differentiable unit.
2. it is different from other networks in respect to the process by which the
weights are calculated during the learning period of the network.
3. training is done in the three stages :
 the feed-forward of input training pattern
 the calculation and backpropagation of the error
 updation of the weight
Working of Backpropagation:
Neural networks use supervised learning to generate output vectors from
input vectors that the network operates on. It Compares generated output to
the desired output and generates an error report if the result does not match
the generated output vector. Then it adjusts the weights according to the bug
report to get your desired output.
Backpropagation Algorithm:

Step 1: Inputs X, arrive through the preconnected path.


Step 2: The input is modeled using true weights W. Weights are usually
chosen randomly.
Step 3: Calculate the output of each neuron from the input layer to the
hidden layer to the output layer.
Step 4: Calculate the error in the outputs
Backpropagation Error= Actual Output – Desired Output
Step 5: From the output layer, go back to the hidden layer to adjust the
weights to reduce the error.
Step 6: Repeat the process until the desired output is achieved.

Parameters :

 x = inputs training vector x=(x 1,x2,…………xn).


 t = target vector t=(t 1,t2……………tn).
 δk = error at output unit.
 δj = error at hidden layer.
 α = learning rate.
 V0j = bias of hidden unit j.

Need for Backpropagation:


Backpropagation is “backpropagation of errors” and is very useful for training
neural networks. It’s fast, easy to implement, and simple. Backpropagation
does not require any parameters to be set, except the number of inputs.
Backpropagation is a flexible method because no prior knowledge of the
network is required.

Types of Backpropagation

There are two types of backpropagation networks.


 Static backpropagation: Static backpropagation is a network designed
to map static inputs for static outputs. These types of networks are
capable of solving static classification problems such as OCR (Optical
Character Recognition).
 Recurrent backpropagation: Recursive backpropagation is another
network used for fixed-point learning. Activation in recurrent
backpropagation is feed-forward until a fixed value is reached. Static
backpropagation provides an instant mapping, while recurrent
backpropagation does not provide an instant mapping.

Advantages:

 It is simple, fast, and easy to program.


 Only numbers of the input are tuned, not any other parameter.
 It is Flexible and efficient.
 No need for users to learn any special functions.

Disadvantages:

 It is sensitive to noisy data and irregularities. Noisy data can lead to


inaccurate results.
 Performance is highly dependent on input data.
 Spending too much time training.
 The matrix-based approach is preferred over a mini-batch.

You might also like