Image Segmentation

Unit VI
Part I
Introduction
• Segmentation is to subdivide an image into its component regions or
objects.

• Segmentation should stop when the objects of interest in an application

have been isolated.

• It is one of the most important steps leading to the analysis of processed

image data.
Introduction
• Every pixel in an image is allocated to one of a number of these
categories.

• A good segmentation is typically one in which:

❑ pixels in the same category have similar grey scale of multivariate values
and form a connected region,

❑ neighboring pixels which are in different categories have dissimilar

values.
Example
Complete vs partial segmentation
• In complete segmentation, ‰
➢ Disjoint regions segmented are uniquely corresponding with objects in
the input image. ‰
➢ Cooperation with higher processing levels which use specific knowledge
of the problem domain is necessary.

• In partial segmentation, ‰
➢ Regions segmented do not correspond directly with image objects.

Totally correct and complete segmentation of complex scenes usually

can't be achieved.

• A reasonable aim is to use partial segmentation as an input to higher level

processing.
How to achieve segmentation?
• Image is divided into separate regions that are homogeneous with respect
to a chosen property such as color, brightness, texture, etc.

• Segmentation algorithms generally are based on 2 basic properties

of gray level values:

1. Discontinuity - The image is partitioned based on abrupt changes in

gray level. Main approach is edge detection isolated points, lines and
edges of image.

2. Similarity - The image is partitioned into homogeneous regions. Main

approaches are thresholding, region growing, region splitting and
merging.
Types
Segmentation methods:

1. Global approaches such as thresholding

2. Edge-based segmentation

3. Region-based segmentation
Detection of Discontinuities

• In discontinuity-based approach, the partitions or sub-division of an image

is based on some abrupt changes in the intensity level of images.

• Here, we mainly interest in identification of isolated points, lined and

edges in an image. To identify these, we use 3x3 Mask operation
Detection of Discontinuities
Detection of Discontinuities

• The response of the mask at any point (x,y) in the image is

Detection of Discontinuities

• The discontinuity-based segmentation can be classified into three

approaches:

• (1) Point detection,

• (2) Line detection, and
• (3) Edge detection.
Point detection

• A point is the most basic type of discontinuity in a digital image.

• This is used to detect isolated spots in an image.
• The gray level of an isolated point will be very different from its neighbors.
• The most common approach to finding discontinuities is to run an (nxn)
mask over each point in the image.

• The mask is as shown in below figure

-1 -1 -1

-1 8 -1

-1 -1 -1
Point detection

-1 -1 -1
-1 8 -1
-1 -1 -1

• The output of the mask operation is usually thresholded.

• The point is detected at a location (x, y) in an image where the mask is
centered.
• We say that an isolated point has been detected if

R=8 f 5 − ( f 1 + f 2 + f 3 + f 4 + f 6 + f 7 + f 8 + f 9 ) > T
i.e |R|>T

• Where R is the response of the mask at any point in the image and
• T is pre defined non-negative threshold value.
Point detection

• It means that isolated point is detected at the corresponding value (x, y).
• This formulation serves to measures the weighted differences between the
center point and its neighbors since the gray level of an isolated point will
be very different from that of its neighbors.
Point detection

Actually any pixel whose value is larger than T after being masked will be
considered as an isolated point.
Detection of lines

This is used to detect lines in an image.

• It can be done using the following four masks:

Detects horizontal lines Detects vertical lines

-1 -1 -1 -1 2 -1
2 2 2 -1 2 -1
-1 -1 -1 -1 2 -1

Detects 45 lines Detects 135 lines

-1 -1 2 2 -1 -1
-1 2 -1 -1 2 -1
2 -1 -1 -1 -1 2
Detection of lines
With the help of lines
detector masks, we can
detect the lines in a specified
direction. For example, we
are interesting in finding all
the lines that are one pixel
thick, oriented at -45°.
Edge detection
➢ Edge detection is the most common approach in gray level discontinuity
segmentation. It is very useful in detecting of discontinuity in an image.

➢ It locates sharp changes in the intensity function.

➢ Edges are pixels where brightness changes abruptly.

➢ An edge is a boundary between two regions having distinct intensity level.

Edge detection
➢ When the image changes from
dark to white or vice-versa.

➢ Magnitude of the first derivative

detects the presence of the edge.

➢ Sign of the second derivative

determines whether the edge pixel
lies on the dark sign or light side.
Figure: (a) Intensity Profile
➢ The changes of intensity, first- (b) First-order derivatives
order derivative and second-order (c) Second-order derivatives
derivative are shown in figure
Edge detection
Edge detection: First-order derivatives
➢ First-order derivatives responds whenever there is discontinuity in intensity
level.
➢ It is positive at the leading edge and negative at the trailing edge.
First-order derivatives: Gradient operator
➢ For a function f(x,y), the gradient of f at coordinates (x',y') is defined as the
vector

➢ Magnitude of vector ∇f (x' , y' ):

➢ Direction of the vector ∇f (x' , y' ):

First-order derivatives: Gradient operator
There is several ways to calculate the image gradient such as
➢ 1. Roberts,
➢ 2. Prewitt and
➢ 3. Sobel operators for computing its value
Prewitt Edge operator
➢ The mask finds the horizontal edges is equivalent to gradient in the vertical
direction and the mask compute the vertical edges is equivalent to gradient
in the horizontal direction.
➢ Using these two masks passing to the intensity image, we can find out and
component at different location in an image.
➢ So, we can find out the strength and direction of edge at that particular
location (x, y).

-1 -1 -1 -1 0 1
0 0 0 -1 0 1
1 1 1 -1 0 1

Masks used for Prewitt Edge operator

Prewitt Edge operator

-1 -1 -1 -1 0 1
0 0 0 -1 0 1
1 1 1 -1 0 1

Masks used for Prewitt Edge operator

Sobel Edge operator
➢ It gives the averaging effect over an image. It considers the effect due to
the spurious noise in the image.
➢ It is preferable over prewitt edge operator because it gives the smoothing
effect and by which we can reduce spurious edge which are generated
because of noise present in the image.

-1 -2 -1 -1 0 1
0 0 0 -2 0 2
1 2 1 -1 0 1

Masks used for Sobel Edge operator

Sobel Edge operator

-1 -2 -1 -1 0 1
0 0 0 -2 0 2
1 2 1 -1 0 1

Masks used for Sobel Edge operator

Second-order derivatives
➢ It is positive at the darker side and negative at the white side.
➢ It is very sensitive to noise present in an image. That’s why it is not
used for edge detection. But, it is very useful for extracting some
secondary information i.e. we can find out whether the point lies on
the darker side or the white side.

Zero-crossing: It is useful to identify the exact location of the edge

where there is gradual transition of intensity from dark to bright
region and vice-versa.

There are several second-order derivative operators:

1. Laplacian operator. The Laplacian mask
2. Canny operator
Laplacian operator
➢ The Laplacian mask is given by:

0 -1 0
-1 4 -1
0 -1 0

Masks used for Laplacian operator

If we consider the diagonal elements:

-1 -1 -1 1 1 1
-1 8 -1 1 -8 1
-1 -1 -1 1 1 1

Masks used for Laplacian operator using 8-connectivity

LOG operator
➢ It is not used for edge detection because it is very sensitive to noise and
also leads to double edge.

➢ But, it is very useful for extracting secondary information.

➢ To reduce the effect of noise, first image will be smooth using the Gaussian
operator and then it is operated by Laplacian operator.

➢ These two operations together is called LoG (Laplacian of Gaussian)

operator
LOG operator
➢ The LoG mask is given by
Canny operator
➢ It is very important method to find edges by isolating noise from the image
before find edges of images, without affecting the features of the edges in
the image and then applying the tendency to find the edges in the image
and the critical value for threshold.
Example
Combined Detection
➢ Detection of combinations of points, lines and edges can be achieved by
using sets of orthogonal masks.
➢ A set of 9 3X3 masks were proposed by Frei and Chen (1977)
Edge linking and boundary detection
➢ The techniques of detecting intensity discontinuities yield pixels lying only
on the boundary between regions.

➢ In practice, this set of pixels seldom characterizes a boundary completely

because if noise, breaks in boundary from non uniform illumination, and
other effects that introduce spurious intensity discontinuities.

➢ Edge detection algorithms are typically followed by linking and other

boundary detection procedures designed to assemble edge pixels into
meaningful boundaries.

➢ We will introduce two linking techniques.

Edge linking and boundary detection
➢ Ideal case: ƒTechniques detecting intensity discontinuities should yield
pixels lying only on edges ( or the boundary between regions). ƒ

➢ Real life: ƒThe detected set of pixels very rarely describes a complete edge
due to effects from: noise, breaks in the edge due to non-uniform
illumination.

➢ Solution: ƒEdge-detection techniques are followed by linking and other

boundary detection procedures which assemble edge pixels into
meaningful boundaries. ƒ

➢ Methods: ƒLocal Processing. ƒ

Global Processing.
Local processing
➢ Analyze the pixel characteristics in a small neighborhood (3x3, 5x5) about
every (x , y) in an image. ƒ

➢ Link similar points to form a edge/boundary of pixels sharing common

properties.

➢ All points that are similar according to predefined criteria are linked,
forming an edge of pixels that share common properties. Similarity
according to:

➢ 1. Strength (magnitude) : The strength of the response of the gradient

operator used to produce the edge pixel.
➢ 2. Direction of the gradient vector
Local processing
➢ 1. Compute the gradient magnitude and angle arrays, M(x,y) and α(x,y), of
the input image, f(x,y) .

➢ 2. Form a binary image, g, whose value at any pair of coordinates (x,y) is

given by:
Local processing
➢ In a small neighborhood, e.g. 3x3, 5x5, all points with common properties
are linked.

➢ A point (x',y') in the neighborhood of (x,y) is linked to the pixel at (x,y) if

both the following magnitude and direction criteria are satisfied.
Local processing
Edge detection and linking
Global Processing via the Hough Transform
➢ Hough transform: a way of finding edge points in an image that lie along a
straight line or curve.

➢ Points are linked by determining whether they lie on a curve of specified

shape. ƒ

➢ Problem: ƒFind subsets of n points that lie on straight lines.

Global Processing via the Hough Transform
➢ Solution: ƒFind all lines determined by every pair of points. ƒFind all
subsets of points close to particular lines. ƒ

➢ Involves:
n(n-1)/2 ~ n2 lines
n(n(n-1))/2 ~ n3 computations for comparing every point to
all lines.
Global Processing via the Hough Transform
➢ Better solution: Hough Transform ƒ

➢ Equation of line passing through point (xi ,yi ):

➢ yi = axi + b (a,b varies) ƒ
➢ But: b = -xi a + yi →
equation of single line on ab plane
Global Processing via the Hough Transform
➢ Performed after Edge detection.

➢ Hough Transform is a technique invented by Paul Hough in 1962 to extract

edge features from an image.

➢ It is a technique to isolate the curves of a given shape / shapes in a given

image.

➢ Classical Hough Transform can locate regular curves like straight lines,
circles, parabolas, ellipses, etc.

➢ Requires that the curve be specified in some parametric form.

➢ Generalized Hough Transform can be used where a simple analytic

description of feature is not possible.
Advantages of Hough Transform
➢ The Hough Transform is tolerant of gaps in the edges.

➢ It is relatively unaffected by noise.

➢ It is also unaffected by occlusion in the image

Image and Parameter Spaces
Thresholding
• Thresholding is one of the most important approaches to image
segmentation.
• If background and object pixels have gray levels grouped into 2 dominant
modes, they can be separated with a threshold easily.

• Thresholding may be viewed as an operation that involves tests against a

function T of the form T=T[x,y,p(x,y),f(x,y)], where f(x,y) is the gray
level of point (x,y), and p(x,y) denotes some local property of this point
such as the average gray level of a neighborhood centered on (x,y).
Thresholding
• Special cases:
• If T depends on
1. f(x,y) only - global threshold
2. Both f(x,y) & p(x,y) - local threshold
3. (x,y) - dynamic threshold

• Multilevel thresholding is in general less reliable as it is difficult to

establish effective thresholds to isolate the regions of interest.
Thresholding
Adaptive thresholding
• The threshold value varies over the image as a function of local image
characteristics.

• Image f is divided into sub images.

• A threshold is determined independently in each subimage.

• If a threshold can't be determined in a subimage, it can be interpolated with

thresholds obtained in neighboring subimages.

• Each sub image is then processed with respect to its local threshold.
Threshold selection based on boundary
characteristics
• A reliable threshold must be selected to identify the mode peaks of a given
histogram.

• This capability is very important for automatic threshold selection in

situations where image characteristics can change over a broad range of
intensity distributions.

• We can consider only those pixels that lie on or near the boundary between
objects and the background such that the associated histogram is well-
shaped to provide a good chance for us to select a good threshold.
Threshold selection based on boundary
characteristics
• The gradient can indicate if a pixel is on an edge or not.

• The Laplacian can tell if a given pixel lies on the dark or light (background
or object) side of an edge.

• The gradient and laplacian can produce a 3-level image.

Threshold selection based on boundary
characteristics
Region-oriented segmentation
• In previous methods, we partition an image into regions by finding
boundaries between regions based on intensity discontinuities.
• Here, segmentation is accomplished via thresholds based on the distribution
of pixel properties, such as intensity or color.
• Basic formulation: Let R represents the entire image which is partitioned
into sub regions R1, R 2,...R n such that
Region-oriented segmentation
• Physical meaning of the formulation: ‰
➢ The segmentation must be complete, i.e. every point must be in a
region. ‰
➢ Points in a region must be connected.
➢ The regions must be disjoint. ‰
➢ It deals with the properties that must be satisfied by the pixels in a
segmented region - for example P(Ri)=true if all pixels in Ri have the same
intensity. ‰
➢ Regions Ri and Rj are different in the sense of predicate P.
Region growing by pixel aggregation
• Start from one seed pixel p located inside region R.
➢ Define a similarity measure S(i; j) for all pixels i and j in the image.
➢ Add adjacent pixel q to pixel p’s region iff S(p; q) > T for some threshold T.
➢ Evaluate the other neighbors of p as above.
➢ We can now consider q as a new seed
➢ Continue until all pixels in the currently investigated neighbourhood do not
satisfy the inclusion criteria.
Region growing by pixel aggregation
Example,
Region growing by pixel aggregation
Example,
Region growing by pixel aggregation
Example,
Region growing by pixel aggregation
Example,
Region growing by pixel aggregation
Problems have to be resolved:

1. Selection of initial seeds that properly represent regions of interest.

2. Selection of suitable properties for including points in the various regions
during the growing process.
3. The formulation of stopping rule.
Region splitting and merging
To subdivide an image initially into a set of arbitrary, disjointed regions and
then merge and/or split the regions in an attempt to satisfy the conditions
stated above.

• A split and merge algorithm is summarized by the following procedure in

which, at each step, we:

(1) split into 4 disjointed quadrants any regions Ri where P(Ri) =false;
(2) merge any adjacent regions Rj and R k for which P(Ri ∪ Rj) =true; and
(3) stop when no further merging or splitting is possible.
Region splitting and merging
Region splitting and merging

(a) The entire image is split into 4 quadrants.

(b) Only the top left region satisfies the predicate so it is not changed, while
the other 3 quadrants are split into sub quadrants.
(c) At this point several regions can be merged, with the exception of the 2
sub quadrants that include the lower part of the object; these do not satisfy
the predicate and must be split further.
Region splitting and merging

• Image segmentation is a preliminary step in most automatic pictorial

pattern-recognition and scene analysis problems.

• The choice of one segmentation technique over another is dictated mostly

by the peculiar characteristics of the problem being considered.
THANK YOU