See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/280083168

Soft Computing: Concepts and Techniques

Book · January 2014

CITATIONS READS
0 5,029

1 author:

Mrutyunjaya Panda
Utkal University
68 PUBLICATIONS   305 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Real Time Product Feedback Review and Analysis Using Apache Technologies and NOSQL Database View project

Big Data Analytics: A social network approach, edited by Mrutyunjaya Panda, Ajith Abraham and Abo Ella Hassanein View project

All content following this page was uploaded by Mrutyunjaya Panda on 11 October 2017.

The user has requested enhancement of the downloaded file.

 SOFT COMPUTING:
CONCEPTS AND TECHNIQUES
 SOFT COMPUTING:
CONCEPTS AND TECHNIQUES

By
M. PANDA M.R. PATRA
Professor and Head, Reader and Head,
Department of Electronics and Department of Computer Science,
Communication Engineering, Berhampur University, Bhanja Vihar
Gandhi Institute for Technological (Odisha)
Advancement, Bhubaneswar
(Odisha)

UNIVERSITY SCIENCE PRESS

(An Imprint of Laxmi Publications Pvt. Ltd.)
BANGALORE ∑ CHENNAI ∑ COCHIN ∑ GUWAHATI ∑ HYDERABAD
JALANDHAR ∑ KOLKATA ∑ LUCKNOW ∑ MUMBAI ∑ RANCHI
NEW DELHI ∑ BOSTON, USA
 Copyright © 2013 by Laxmi Publications Pvt. Ltd. All rights reserved. No
part of this publication may be reproduced, stored in a retrieval system, or
transmitted in any form or by any means, electronic, mechanical, photocopying,
recording or otherwise without the prior written permission of the publisher.

Published by :
UNIVERSITY SCIENCE PRESS
(An Imprint of Laxmi Publications Pvt. Ltd.)
113, Golden House, Daryaganj,
New Delhi-110002
Phone : 011-43 53 25 00
Fax : 011-43 53 25 28
www.laxmipublications.com
[email protected]

Price : < 95.00 Only. First Edition: 2013

OFFICES

Bangalore 080-26 75 69 30 Chennai 044-24 34 47 26

Cochin 0484-237 70 04, 405 13 03 Guwahati 0361-251 36 69, 251 38 81
Hyderabad 040-24 65 23 33 Jalandhar 0181-222 12 72
Kolkata 033-22 27 43 84 Lucknow 0522-220 99 16
Mumbai 022-24 91 54 15, 24 92 78 69 Ranchi 0651-2204464, 221 47 64
USC-9664-095-SOFT COMPUTING-PAN C—
Typesetted at : Shubham Composer, New Delhi. Printed at :
 CONTENTS
CHAPTER 1. Artificial Intelligent System: An Introduction 1–13
1.1 Introduction 1
1.2 History of AI 1
1.3 Pros and Cons of AI 2
1.4 Basic Tools of Soft Computing 2
1.5 Approximations of Multivariate Functions 5
1.6 Non-linear Error Surface and Optimization 8
1.7 Non-linear Optimization 10
CHAPTER 2. Fuzzy Logic Systems 14–48
2.1 Fuzzy vs Crisp 14
2.2 Set 14
2.3 Basic Fuzzy Set Operations 16
2.4 Fuzzy Systems 26
2.5 Tautology 30
2.6 Implication of Propositional Logic 31
2.7 Reasoning with Proposition 32
2.8 Predicate Logic 33
2.9 Fuzzy Reasoning Methods 35
2.10 Fuzzy Composition 36
2.11 Fuzzy Quantifiers 36
2.12 Fuzzy Systems 36
2.13 Fuzzy Inference System (FIS) 43
2.14 Adaptive Neuro Fuzzy Inference System (ANFIS) 46
2.15 Application to Pattern Recognition 47
CHAPTER 3. Neural Networks 49–81
3.1 What is Neural Network? 49
3.2 The Biological Neuron 49
3.3 A Model of an Artificial Neuron 50
3.4 Artificial Neural Network 53
3.5 Single Layer ANN 57
3.6 Perceptron 57
3.7 Multilayer Perceptron (MLP) 58
3.8 The Adaptive Linear Element (ADALINE) 60
3.9 Networks with Linear Activation Functions: The Delta Rule 60
3.10 Training Multilayer Perceptron Network 62
3.11 Error Back Propagation Algorithm 63
3.12 Radial Basis Function 66
(v)
 (vi)

3.13 Least-squares 68
3.14 Kohonen Self-organizing Map Networks 69
3.15 Learning Vector Quantization 72
3.16 Recurrent Neural Network 75
3.17 Simulated Annealing Neural Network 76
3.18 Neural Networks and Pattern Recognition 77
3.19 Neural Networks and Control 78
CHAPTER 4. Genetic Algorithms 82–92
4.1 Introduction 82
4.2 History of GA 82
4.3 Crossover Operators 83
4.4 Mutation 85
4.5 Method of Representation 87
4.6 Methods of Selection 87
4.7 Genetic Programming 87
4.8 Traditional Optimization Methods 90
4.9 Advantages 90
4.10 Limitations 91
4.11 Some Examples of GAs 91
References 93–95
Index 96–97
 FOREWORD
Mrutyunjaya Panda is an experienced teacher as well as a scientist, does a lot of research work
in the field of soft computing and network security. He has contributed many things in this book
which are outcome of his teaching experience and research work so that Neuro-Fuzzy and soft
computing demonstrates his knowledge in this field, his mastery and insightfulness with expository
skills. His coauthor Dr. Manas Ranjan Patra has made an important contribution in writing the
text using his extensive experience in dealing with real world problems.
The soft computing concepts and techniques are aimed at an accommodation with the pervasive
imprecision of the real world problems so as to exploit the tolerance for imprecision, uncertainty,
and robustness. However, the real role model for soft computing is the Human brain.
This book Soft Computing: Concepts and Techniques has been designed for undergraduate students
computer science and other branches of engineering. The book is enriched with a wealth of
information which is lucidly presented, well organized and illustrated by many examples. This
will help the reader to acquire the needed background in soft computing for further work in
designing many efficient intelligent systems.
Dr. Subrata Chattopadhyay,
Department of Electrical Engineering
NITTTR, Kolkata

(vii)
 PREFACE
Soft computing (SC), is an innovative approach to computing which parallels the remarkable ability
of the human mind to reason and learn in an environment of uncertainty and imprecision. Soft
computing techniques having the roots in artificial intelligence (AI) consist of fuzzy logic (FL) neural
network (NN) and evolutionary algorithms (EA).
There are many potential applications of soft computing methodologies, which includes pattern
recognition, adaptive control system, robotics, data mining, and so on.
This book has been written for undergraduate students and others, who are interested in using
soft computing techniques. It has been developed as a textbook for one semester course at univer-
sity for students pursing computer science and in almost all other branches of engineering. During
the course of teaching soft computing, I found that no textbook covers the complete syllabi of the
soft computing. In this book, we have attempted to cover all the syllabi along with many examples
and their applications to pattern recognition and control.
The text is divided into 4 chapters. Chapter 1 gives an overview of artificial intelligence with their
Pros and Cons, followed by the various tools of soft computing needed to understand the concept.
Non-Linear error surface and optimization techniques are also discussed to understand the usefulness
of these in soft computing.
Chapter 2 covers the fundamentals of fuzzy logic. It discusses some insights to the fuzzy relations,
fuzzy inferencing along with examples that describes the real world applications. It also discusses
about the various defuzzification methods.
Chapter 3 introduces neural networks. It basically discusses various neural network architectures,
such as feed-forward (FFNN), backpropagation (BPN), radial basis function (RBF), self-organizing
maps (SOM), linear vector quantization (LVQ) and hybrid neuro-fuzzy system (ANFIS), with many
solved examples to understand the concept of soft computing.
Finally, in Chapter 4, fundamentals of genetic algorithms (GAs) and genetic programming (GP)
are covered with several examples. It presents different types of genetic representation, crossover
operators and selection mechanism, along with various optimization methods like simulated
annealing (SA), hill climbing (HL), etc. The chapter concludes with possible applications to robotics,
data mining, and pattern recognitions.
—Authors

(ix)
 ACKNOWLEDGEMENTS
We would like to thank Dr. S. P. Panda, Chairman, GGI, Gunupur and Dr. C. D. Panda, Secretary,
GGI, Gunupur for their constant support, Prof. K. Govinda Rajulu, Former Head, ECE, GIET, Gunupur
for his early motivation, and Dr. R. P. Panda, Head, ECE, VSSUT, Burla for his valuable suggestions
in writing this book. Further, we would express our sincere thanks to all others who have directly
and/or indirectly helped us in bringing out this book in the present form in a short span of time.

—Authors

(x)
 ARTIFICIAL INTELLIGENT SYSTEM: AN INTRODUCTION 1

1
ARTIFICIAL INTELLIGENT
SYSTEM: AN INTRODUCTION
1.1 INTRODUCTION

Artificial intelligence, or AI for short, is a combination of computer science, physiology, and philosophy.
AI is a broad topic, consisting of different fields, form machine vision to expert systems. The elements
that the fields of AI have in common is the creation of machines that can “think”.
In order to classify machines as “thinking”, it is necessary to define intelligence. Research into
the areas of learning, of language, and of sensory perception have aided researchers in building
intelligent machines. One of the most challenging approaches facing experts is building system
that mimic the behaviour of the human brain, made up of billions of neurons, and arguably the
most complex matter in the universe. As per Alan tuning, a British computer scientist, a computer
would deserves to be called intelligent if it could deceive a human into believing that it was human.
The beginning of AI started from an eminent mathematician “Boole” and others; theorizing on
principles that were used as the foundation of AI logic. AI has always been on the pioneering end of
computer science. Advanced level computer languages, as well as computer interfaces and word-
processors owe their existence to the research into artificial intelligence, which will set the trend in
the future of computing.
As a prelude, we shall provide a bird’s-eye-view of relevant intelligent system approaches,
along with bits of their history and discuss the features of soft computing paradigms.

1.2 HISTORY OF AI

Evidence of artificial intelligence folklore can be traced back to ancient Egypt, but the technology
finally became available to create machine intelligence, after the development of the electronic
computer in 1941. Through its short modern history, advancement in the fields of AI have been
slower that first estimated, progress continues to be made. From its birth 5 decades ago, there have
been a variety of AI programs, which have impacted other technological advancements. The history
of AI is shown below in Figure 1.1.
Year History of AI Comments
1941 First Electronic computer —
1949 First commercial, stored program computer The birth of
1956 Dartmouth conference and logic theorist developed AI

1
2 SOFT COMPUTING: CONCEPTS AND TECHNIQUES

1958 LISP language developed —

1963 Start of DOD’s advanced Research Projects
1968 Microworld program, SHRDLU created. —
1970 First expert system —
1972 PROLOG language revealed —
1986 AI based hardware sells US $ 425 million to companies —
1991 AI system beats human chess master —
Fig. 1.1 The history of artificial intelligence

1.3 PROS AND CONS OF AI

Pros of AI : Emotions that often intercept rational thinking of a human being are not a hindrance for
artificial thinkers. Lacking the emotional side, robots can think logically and take the right decisions.
Sentiments are associated with moods that affect human efficiency. This is not the case with machines
with artificial intelligence.
Thus artificial intelligence can be utilized in the completion of repetitive and time-consuming tasks
efficiently. Intelligent machines can be used to perform certain dangerous tasks. Machines equipped
with AI can be mode to thoughtfully plan towards the fulfilment of tasks and accordingly adjust
their parameters such as speed and time. They can be made to act quickly, unaffected by anything
like emotion and take the tasks toward perfection.
Cons of AI : The first concern regarding the application of AI is about ethics and moral values. Is
it ethically correct to create replicas of human beings? Do our moral values allow us to recreate
intelligence? Intelligence is after all a gift of nature. It may not be right to install it into a machine to
make it work for our benefit.
Secondly, imagine robots working in hospitals. Do you picture them showing care and concern
towards the patients? Human beings are emotional intellectuals. They think and feel. Their feelings
guide their thoughts.
Eventually, it is up to us whether to stand by AI or warn ourselves of the likely disaster that it
may lead to. In our opinion, there is no ideal replacement for human beings. AI can help alleviate
the difficulties faced by man but can never be “human”.

1.4 BASIC TOOLS OF SOFT COMPUTING

Soft computing is considered to be an important tool to perform several computing operations that
include neural networks, fuzzy logic, models, approximate reasoning, and evolutionary algorithms
such as genetic algorithms and simulated annealing. The soft computing allows to incorporate human
knowledge effectively, deal with uncertainty, imprecision, and learns to adapt to unknown or
changing environment for better performance. As soft computing does not perform much symbolic
manipulations, we can therefore view it as a new discipline that complements conventional artificial
intelligence (AI) approaches, and vice-versa.

1.4.1 Basics of Neural Network

The human brain is a collection of about 10 billion interconnected neurons. Each neuron is a cell
that uses biochemical reactions to receive, process, and transmit information.
 ARTIFICIAL INTELLIGENT SYSTEM: AN INTRODUCTION 3

Inspired by the biological nervous system, a novel non algorithmic approach such as artificial
neural networks; have been explored by many scientist and researchers for information processing.
Thus, artificial neural network is an information processing paradigm which attempts to simulate
the functionality of human brain and model non linear statistical data.
Then the question arises: “what is the need to simulate human brain when we have computers
which can perform millions of numerical calculations within a fraction of second?”
The answer to this is, even though a computer can store a huge amount of data and can perform
numerical calculations efficiently, there are some problems which it struggles to solve.
Let us take an example, we have text based search engines that are capable of searching millions
of files across the internet for a text key. Thinking in similar lines, imagine an image searching
application, where the user is given some space to draw an image and the application should search
and show all images similar to the one drawn. If the user draws a tree, the search engine should
return all images having a tree.
This image search belongs to a broad class of similar problems, called “Pattern Recognitions”
problems. Optical character recognition, hand writing recognition, face recognition, speech
recognition, image processing, signature recognition and speaker recognition are different forms of
pattern recognition.
Though a computer can store huge amounts of data, it does not know how to identify, organize,
understand and interpret the data. On the other hand, human brain is incredibly efficient in
identifying patterns. Just by looking at one’s face, we can recognize who the person is (if we know him),
but it is not so easy for a computer. This is because, there is no algorithm to map a picture
(or a video) accurately to a person’s identity. It is not possible mathematically to model these kinds
of problems. This makes it impossible to solve such problems using conventional problem solving
techniques. If we want to solve pattern recognition, we have to somehow simulate human brain.
The concept of artificial neural network is an attempt to the same.
Artificial neural networks take a different problem solving approach than that of conventional
computers. They connot be programmed to solve a specific problem. They learn by example. This gives
them a remarkable property of deriving meaningful information from complicated data and allows
them to extract patterns and detect trends that are too complex to be noticed by other computing techniques.

1.4.1.1 Applications
Artificial neural networks are extensively used in applications involving:
(i) Function Modelling
(ii) Biometric Pattern Recognition
(iii) Classification
(iv) Prediction
(v) Forecasting , and
(vi) Data clustering problems.

1.4.2 Fuzzy Logic

Consider the following statement, which is the base on which fuzzy logic is built:
“As the complexity of a system increases, it becomes more difficult and eventually impossible to
make a precise statement about its behaviour, eventually arriving at a point of complexity where
the fuzzy logic method borns in humans is the only way to get at the problem”. (Originally identified
by Lofti A. Zadeh, Ph.D, University of California, Berkeley).
4 SOFT COMPUTING: CONCEPTS AND TECHNIQUES

Fuzzy logic is used in system control and analysis design, because it shortens the time for
engineering development and sometimes, in the case of highly complex system, is the only way to
solve the problem.
Human beings have the ability to take in and evaluate all sorts of information from the physical
world they are in contact with and to mentally analyze, average and summarize all this input data
into an optimum course of action. All living things do this, but human do it more and do it better
and have become the dominant species of the planet.
If we think about it, much of the information you take in is not very precisely defined, such as
evaluation of the behaviour of a vehicle entering form a side of the street and the likelihood of the
vehicle pulling in front of you. We call this as fuzzy input. However, some of our “input” is reasonably
precise and non-fuzzy such as speedometer readings. The processing of all these informations are
not very precisely definable. We call this fuzzy processing. Fuzzy Logic theorists would call it
using fuzzy algorithms (algorithm is another word for procedure or program, as in a computer
program). So, fuzzy logic is the way the human brain works, and we can mimic this in machines,
to perform somewhat like humans, not to be confused with artificial intelligence, where the so far
unattainable goal is for machines to perform EXACTLY like humans ( see Forbes magazine, December
2009, digital tools , “why computer can’t mimic the Brain: our gray matter is far too complex for
machine to simulate”.

1.4.2.1 The Fuzzy Logic Method

Fuzzy logic control and analysis systems may be electromechanical in nature, or concerned only with
data, for example economic data, in all cases guided by “if- then” rules stated in human language.
Therefore, the fuzzy logic analysis and control method is:
(a) Receiving of one or a large number of measurements or other assessment of conditions existing
in some system we wish to analyze or control.
(b) Processing all these inputs according to human based, fuzzy “IF-THEN” rules, which can be
expressed in plain language words.
(c) Averaging and weighting the resulting outputs form all the individual rules into one single
output decision or signal which decides what to do or tells a controlled system what to do.
The output signal eventually arrived at, is a precise appearing, defuzzified “crisp” value. The
details about the fuzzy logic and fuzzy set theory are described in Chapter-2 .

1.4.3 Evolutionary Computation

Evolutionary Computation is an area of research within computer science, which draws inspiration
from the process of natural evolution. Evolutionary computation offers practical advantages to the
researchers facing difficult optimization problems. These advantages are multi-fold, including the
simplicity of the approach, its robust response to changing circumstances, its flexibility and many
other facets. The evolutionary approach can be applied to problems where heuristic solutions are
not available or generally led to unsatisfactory results. Thus evolutionary computing is needed for
developing automated problem solvers, where the most powerful natural problem solvers are
human brain and evolutionary process (that created the human brain). Designing the problem
solvers based on human brain leads to the field of “neuro computing”, while the second one leads
to “evolutionary computing”. The algorithms involved in evolutionary computing are termed as
evolutionary algorithms (EA).
 ARTIFICIAL INTELLIGENT SYSTEM: AN INTRODUCTION 5

The various applications of evolutionary computing may include:

(a) Bio Informatics
(b) Numerical Combinatorial Optimization
(c) System Modeling and Identification
(d) Planning and Control
(e) Engineering Design
(f) Data Mining
(g) Machine Learning, and
(h) Artificial Life
More discussion about EA is provided in chapter-4.

1.5 APPROXIMATIONS OF MULTIVARIATE FUNCTIONS

1.5.1 Univariate Functions

Problems with a single variable are called univariate. The univariate optimum for Y = f(X) occurs
at points where the first derivative of f(X) with respect to x (i.e., f’(X)) equals to zero. However,
points which have zero first derivatives do not necessarily constitute a minimum or maximum.
The second derivative is used to discover character of a point. Points at which a relative minimum
occurs have a positive second derivative at that point, while relative maximum occurs at points
with a negative second derivative. Zero second derivatives are in conclusive.
It is also important to distinguish between local and global optima. A local optimum arises
when one finds a point whose value in the case of a maximum exceeds that of all surrounding
points but may not exceed that of distant points. The second derivative indicates the shape of
functions and is useful in indicating whether the optimum is local a global. The second derivative
is the rate of change in the first derivative. If the second derivative is always negative (or positive)
that implies that any maximum (or minimum) found is a global result.
Consider a maximization problem, with a negative second derivative for which f’(X*) = 0. This
means the first derivative was > 0 for X < X* and was < 0, for X > X*. The function can never rise
when moving away from X* because of the sign of the second derivative. An everywhere positive
second derivative indicates a global minimum, will be found if f’(X*) = 0, while a negative indicates
a global maximum.

1.5.2 Multivariate Functions

The univariate optimization results have multivariate analogues. In the multivariate case, partial
derivatives are used, and a set of simultaneous conditions is established. The first and second
derivatives are again key to the optimization process, excepting now that a vector of first derivatives
and a matrix of second derivatives is involved.
There are several terms to consider in this case. First, the gradient vector, Lx f(XE), is the vector of
first order partial derivatives of a multivariate function w.r.t. each of the variables evaluated at the
point XE.
È ∂f (X *) ˘
—f (X *k ) = Í ˙,
ÎÍ ∂ X k ˚˙
∂f (X E )
where stands for the partial derivatives of f(X) w.r.t. Xk evaluated at XE, and XE depicts
∂X K
XIE, X2E, …, XmE. The second derivatives constitute the Hessian matrix,
View publication stats