CONTENTS

CHAPTER 1. MAKE IMAGES COME ALIVE WITH SCIKITIMAGE..................................4

THEORIES...................................................................................................................................4
1.1. Image processing.............................................................................................................4
1.1.1. What is image processing?.......................................................................................4
1.1.2. Purposes....................................................................................................................4
1.1.3. Intro to scikit-image.................................................................................................4
1.1.4. What is an image?....................................................................................................4
1.1.5. Images in scikit-image..............................................................................................5
1.1.6. RGB channels...........................................................................................................5
1.1.7. Grayscaled images....................................................................................................5
1.1.8. RGB vs Grayscale....................................................................................................5
1.1.9. Visualizing images in the course..............................................................................6
1.2. NumPy for images...........................................................................................................7
1.2.1. NumPy for images....................................................................................................7
1.2.2. Images as NdArrays.................................................................................................7
1.2.3. Colors with NumPy..................................................................................................7
1.2.4. Shapes and size.........................................................................................................8
1.2.5. Flipping images........................................................................................................9
1.2.6. What is a histogram?................................................................................................9
1.2.7. Color histograms......................................................................................................9
1.2.8. Applications of histograms.....................................................................................10
1.2.9. Histograms in Matplotlib........................................................................................10
1.3. Thresholding..................................................................................................................11
1.3.1. Thresholding...........................................................................................................11
1.3.2. Categories...............................................................................................................12
1.3.3. Try more thresholding algorithms..........................................................................12
1.3.4. Optimal thresh value...............................................................................................13

EXERCISE:................................................................................................................................15
Exercise 1:...............................................................................................................................15
Exercise 2: RGB to Grayscale.................................................................................................16
Exercise 3: Flipping out.............................................................................................................17
Exercise 4: Histograms..............................................................................................................18
Exercise 5: Apply global thresholding....................................................................................19
Exercise 6: When the background isn't that obvious...............................................................20
Exercise 7: Trying other methods............................................................................................21
Exercise 8: Apply thresholding...............................................................................................22

1
CHAPTER 2. FILTERS.............................................................................................................24

THEORIES:................................................................................................................................24
Filtering...................................................................................................................................24
2.1. Filters.............................................................................................................................24
2.1.1. Neighborhoods.......................................................................................................24
2.1.2. Edge detection........................................................................................................24
2.1.3. Comparing plots.....................................................................................................25
2.1.4. Gaussian smoothing................................................................................................26
2.2. Contrast.........................................................................................................................27
2.2.1. Contrast enhancement............................................................................................27
2.2.2. Contrast...................................................................................................................28
2.2.3. Enhance contrast.....................................................................................................28
2.2.4. Histogram equalization...........................................................................................29
2.2.5. Adaptive Equalization............................................................................................30
2.2.6. Contrastive Limited Adaptive Equalization...........................................................30
2.2.7. CLAHE in scikit-image..........................................................................................31
2.3. Transformations............................................................................................................31
2.3.1. Why transform images?..........................................................................................31
2.3.2. Rotating..................................................................................................................32
2.3.3. Rescaling................................................................................................................33
2.3.4. Aliasing in digital images.......................................................................................33
2.3.5. Resizing..................................................................................................................34
2.4. Morphology:..................................................................................................................36
2.4.1. Binary images.........................................................................................................36
2.4.2. Morphological filtering..........................................................................................36
2.4.3. Morphological operations.......................................................................................36
2.4.4. Structuring element................................................................................................36
2.4.5. Shapes in scikit-image............................................................................................37
2.4.6. Erosion in scikit-image...........................................................................................37
2.4.7. Dilation in scikit-image..........................................................................................38

EXERCISE:................................................................................................................................39
Exercise 1: Edge detection......................................................................................................39
Exercise 2: Blurring to reduce noise.............................................................................................40
Exercise 3:...............................................................................................................................42
Exercise 4: Medical images.....................................................................................................42
Exercise 5: Aerial image............................................................................................................43
Exercise 6: Let's add some impact and contrast......................................................................44
Exercise 7: Aliasing, rotating and rescaling............................................................................46
Exercise 8: Enlarging images..................................................................................................46
Exercise 9: Proportionally resizing.........................................................................................48
Exercise 10: Morphology........................................................................................................49
Exercise 11: Improving thresholded image.............................................................................50
2
CHAPTER 3. IMAGE RESTORATION...................................................................................51

THEORIES.................................................................................................................................51
3.1. Image restoration...........................................................................................................51
3.1.1. Restore an image....................................................................................................51
3.1.2. Image reconstruction..............................................................................................51
3.1.3. Masks......................................................................................................................53
3.2. Noise..............................................................................................................................54
3.2.1. Noise.......................................................................................................................54
3.2.2. Apply noise in scikit-image:...................................................................................55
3.2.3. Reducing noise.......................................................................................................55
3.3. Superpixels & segmentation..........................................................................................57
3.3.1. Segmentation..........................................................................................................57
3.3.2. Image representation..............................................................................................58
3.3.3. Superpixels.............................................................................................................59
3.3.4. Segmentation..........................................................................................................59
3.3.5. Unsupervised segmentation....................................................................................59
3.4. Finding contours............................................................................................................61
3.4.1. Finding contours.....................................................................................................61
3.4.2. Find contours using scikit-image............................................................................62
3.4.3. Constant level value...............................................................................................63
3.4.4. The steps to spotting contours................................................................................63
3.4.5. A contour's shape....................................................................................................64

EXERCISE.................................................................................................................................64
Exercise 1: restore a damaged image......................................................................................64
Exercise 2: Removing logos.......................................................................................................65
Exercise 3: Noise.....................................................................................................................66
Exercise 4: Reducing noise........................................................................................................67
Exercise 5: Reducing noise while preserving edges........................................................................68
Exercise 6: Number of pixels......................................................................................................69
Exercise 7: Superpixel segmentation............................................................................................70
Exercise 8: Contouring shapes....................................................................................................71
Exercise 9: Find contours of an image that is not binary..................................................................72
Exercise 10: Count the dots in a dice's image................................................................................72

CHAPTER 4. FINDING THE EDGES WITH CANNY...........................................................74

THEORIES:................................................................................................................................74
4.1. Finding the edges with Canny.......................................................................................74
4.1.1. Detecting edges......................................................................................................74
4.1.2. Edge detection........................................................................................................74
4.1.3. Canny edge detector...............................................................................................75
4.2. Right around the corner.................................................................................................76
4.2.1. Corner detection.....................................................................................................76
3
 4.2.2. Points of interest.....................................................................................................76
4.2.3. Corners...................................................................................................................77
4.2.4. Matching corners....................................................................................................77
4.2.5. Harris corner detector.............................................................................................78
4.3. Face detection................................................................................................................80
4.3.1. Face detection use cases.........................................................................................80
4.3.2. Detecting faces with scikit-image..........................................................................81
4.3.3. Let's try detecting faces..........................................................................................81
4.4. Real-world applications.................................................................................................84
4.4.1. Applications............................................................................................................84
4.4.2. Privacy protection...................................................................................................84
4.4.3. More cases..............................................................................................................87

EXERCISE:................................................................................................................................87
Exercise 1: Finding the edges with Canny..............................................................................87
Exercise 2: Less edgy................................................................................................................88
Exercise 3:...............................................................................................................................89
Exercise 4: Less corners.............................................................................................................91
Exercise 5: Face detection.......................................................................................................92
Exercise 6: Multiple faces..........................................................................................................93
Exercise 7: Segmentation and face detection..........................................................................94
Exercise 8: Privacy protection.................................................................................................96
Exercise 9: Help Sally restore her graduation photo...............................................................97

4
CHAPTER 1. MAKE IMAGES COME ALIVE WITH SCIKITIMAGE

THEORIES
1.1. Image processing
1.1.1. What is image processing?

Image processing is a method to perform operations on images, in order to enhance them

or extract useful information, analyze it and make decisions.
By quantifying the information in images, we can make calculations. Image processing is
a subset of computer vision.
Applications: There is a wide range of applications. Such as the analysis of medical
images, artificial intelligence, image restoration, surveillance and many more.

1.1.2. Purposes

The purpose of image processing can be divided into five groups. Visualization: observe
objects that are not visible. Image sharpening and restoration: to create a better image. Image
retrieval: to seek for the image of interest. Measurement of pattern: to measure objects. And
Image Recognition: to distinguish objects in an image

1.1.3. Intro to scikit-image

scikit-image is an image processing library in Python that is easy to use. scikit-image

makes use of Machine Learning with built-in functions, and can perform complex operations
on images with just a few functions. We will use scikit-image throughout this course

1.1.4. What is an image?

Let's get started with images! A digital image is an array, or a matrix, of square pixels
(picture elements) arranged in columns and rows: in other words, a 2-dimensional matrix.
These pixels contain information about color and intensity. Here's an example of the
matrix for a 2D grayscale image. Here we can see that the first image is a pixelated image. The
numbers that we see on top of the following image next to the first one correspond to the
intensity of each pixel in the image. So at the end, an image can be treated as an intensities
matrix.

5
1.1.5. Images in scikit-image

There are some testing-purpose images provided by scikit-image, in a module called data.
If we want to load a colored image of a rocket, we can do so by: Importing data from skimage.
And then from data, call a method named rocket.
Code:
from skimage import data
rocket_image = data.rocket()

1.1.6. RGB channels

2-dimensional color images are often represented in RGB—3 layers of 2-dimensional

arrays, where the three layers represent Red, Green and Blue channels of the image

1.1.7. Grayscaled images

Grayscale images only have shades of black and white. Often, the grayscale intensity is
stored as an 8-bit integer giving 256 possible different shades of gray. Grayscale images don't
have any color information

6
1.1.8. RGB vs Grayscale

RGB images have three color channels, while grayscaled ones have a single channel. We
can convert an image with RGB channels into grayscale using the function rgb2gray() provided
in the color module. We can also turn grayscale to RGB using gray2rgb().
Code:
from skimage import color
grayscale = color.rgb2gray(original)
rgb = color.gray2rgb(grayscale)

1.1.9. Visualizing images in the course

During the course, we'll usually use a preloaded function called show image to display the
images using Matplotlib. This way we can focus on image processing code. This is what the
show image function looks like.
Code:
def show_image(image, title='Image', cmap_type='gray'):
plt.imshow(image, cmap=cmap_type)
plt.title(title)
plt.axis('off')
plt.show()
So if we want to show an image that we have converted to grayscale, we just need to pass
the image as the first parameter of the "show_image" function and the title "Grayscale" as the
second parameter. We will then see the image Grayscale displayed in the console output.
Code:
from skimage import color
grayscale = color.rgb2gray(original)
7
 show_image(grayscale, "Grayscale")

1.2. NumPy for images

1.2.1. NumPy for images

With NumPy, we can practice simple image processing techniques, such as flipping
images, extracting features, and analyzing them.

1.2.2. Images as NdArrays

Imagine that we have an image and we load it using matplotlib's imread() function. If you
check it's type, using the type() python function, you can see that is a numpy ndarray object.
Because images can be represented by NumPy multi-dimensional arrays (or "NdArrays"),
NumPy methods for manipulating arrays work well on these images
Code:
# Loading the image using Matplotlib
madrid_image = plt.imread('/madrid.jpeg')
type(madrid_image)

1.2.3. Colors with NumPy

Remember that a color image is a NumPy array with an third dimension for color
channels. We can slice the multidimensional array and obtain these channels separately.
Here we can see the individual color intensities along the image. For example, we obtain
the red color of an image by keeping the height and width pixels and selecting only the values
of the first color layer. Here we use Matplotlib to display them with the default colormap. We
can observe the different intensities in their tone.
Code:
# Obtaining the red values of the image
red = image[: , : , 0]
# Obtaining the green values of the image
green = image[: , : , 1]
# Obtaining the blue values of the image
blue = image[: , : , 2]

8
 We can also display them using the gray colormap, specifying it with the cmap attribute,
of the imshow function. We can still see the different intensities along the images, one for each
color. The red, green and blue.
Code:
plt.imshow(red, cmap="gray")
plt.title('Red')
plt.axis('off')
plt.show()

1.2.4. Shapes and size

Just like with NumPy arrays, we can get the shape of

images. This Madrid picture is 426 pixels high and 640
pixels wide. It has three layers for color representation: it's
an RGB-3 image. So it has shape of (426, 640, 3).
Code:
# Accessing the shape of the image
madrid_image.shape
Sizes: And a total number of pixels of 817920
Code:
# Accessing the shape of the image
madrid_image.size

9
1.2.5. Flipping images

Vertically: We can flip the image vertically by using the flipud() method. We are using
the show_image() function to display an image.
Horizontally: You can flip the image horizontally using the fliplr() method.
Code:
# Flip the image in up direction
vertically_flipped = np.flipud(madrid_image)
show_image(vertically_flipped, 'Vertically flipped image')

# Flip the image in left direction

horizontally_flipped = np.fliplr(madrid_image)
show_image(horizontally_flipped, 'Horizontally flipped image')

1.2.6. What is a histogram?

The histogram of an image is a

graphical representation of the
amount of pixels of each intensity
value. From 0 (pure black) to
255(pure white). The first image is
really dark, so most of the pixels have
a low intensity, from 0 to 50. While
the second one, it's lighter and has
most of the pixels close to 200 and
255.

10
1.2.7. Color histograms

We can also create histograms from RGB-3 colored images. In this case each channel:
red, green and blue will have a corresponding histogram.

1.2.8. Applications of histograms

We can learn a lot about an image by just looking at its histogram. Histograms are used to
threshold images (an important topic in computer vision that we will cover later in this course),
to alter brightness and contrast, and to equalize an image (which we will also cover later in this
course).

1.2.9. Histograms in Matplotlib

Matplotlib has a histogram method. It takes an input array (frequency) and bins as
parameters. The successive elements in bin array act as the boundary of each bin. We obtain
the red color channel of the image by slicing it. We then use the histogram function. Use ravel
to return a continuous flattened array from the color values of the image, in this case red. And
pass this ravel and the bins as parameters. We set bins to 256 because we'll show the number of
pixels for every pixel value, that is, from 0 to 255. Meaning you need 256 values to show the
histogram.
Visualizing histograms with Matplotlib: So to display it, once we obtain the blue color of
the image and use the hist method, by passing the array and the bins to put in the graphic. Plot
it using plt.show().
Code:
# Red color of the image
red = image[: , : , 0]
# Obtain the red histogram
plt.hist(red.ravel(), bins=256)
blue = image[: , : , 2]
plt.hist(blue.ravel(), bins=256)
plt.title('Blue Histogram')
11
 plt.show()

1.3. Thresholding
1.3.1. Thresholding

Thresholding is used to partition the background and foreground of grayscale images, by

essentially making them black and white. We compare each pixel to a given threshold value. If
the pixel is less than that value, we turn it white. If it's greater; we turn it black.
Thresholding is the simplest method of image segmentation, a topic we will cover in
more detail later. Thresholding let us isolate elements and is used in object detection, facial
recognition, and other applications.
It works best in high-contrast grayscale images. To threshold color images, we must first
convert them to grayscale.
Let's see how to apply it. Once an image is loaded, we need to set the thresh value.
Temporarily set it to 127, mid point between 0 and 255. We apply thresholding to an image by
using the greater than operator followed by the thresh. Finally, show the thresholded image
using show_image().
Code:
# Obtain the optimal threshold
value
thresh = 127
# Apply thresholding to the image
binary = image > thresh
# Show the original and
thresholded
show_image(image, 'Original')
show_image(binary, 'Thresholded')

12
 Inverted thresholding: We can also apply inverted thresholding, which is just inverting
the color. We apply it just as we do in thresholding, except we use "<=" operator instead of
">". Here the resulting image is also binary but the background is black and foreground white.
Code:
# Obtain the optimal threshold value
thresh = 127
# Apply thresholding to the image
binary = image <= thresh
# Show the original and thresholded
show_image(image, 'Original')
show_image(inverted_binary,'Inverted
Thresholded')

1.3.2. Categories

There are two categories of thresholding in scikit-image: Global or histogram-based,

which is good for images that have relatively uniform backgrounds. Adaptive or local, which is
best for images where the background is not easily differentiated, with uneven background
illumination. Note that local is slower than global thresholding. Here we have an image to
compare. In this case, it seems that local is without a doubt the best option.

1.3.3. Try more thresholding algorithms

What if I want to try more algorithms? Well scikit-image includes a function that
evaluates several global algorithms, so that you can choose the one that gives you best results:
the try_all_threshold function from filters module. Here we import it, use the function by
passing the image and set verbose to False so it doesn't print function name for each method.
And show results
Code:
from skimage.filters import try_all_threshold

13
 # Obtain all the resulting images
fig, ax = try_all_threshold(image, verbose=False)
# Showing resulting plots
show_plot(fig, ax)
It will use seven global algorithms, so here we see first the original image followed by the
resulting images of the thresholding methods. We cover only the otsu. So this is a easy way the
rest!

1.3.4. Optimal thresh value

When the background of an image seems uniform, global thresholding works best.
Previously, we arbitrarily set the thresh value, but we can also calculate the optimal value. For
that we import the threshold_otsu() function from filters module. Then obtain the optimal
global thresh value by calling this function. Apply the local thresh to the image. And that's it!
Code:
# Import the otsu threshold function
from skimage.filters import threshold_otsu
# Obtain the optimal threshold value
thresh = threshold_otsu(image)
# Apply thresholding to the image
binary_global = image > thresh
# Show the original and binarized image
show_image(image, 'Original')
show_image(binary_global, 'Global thresholding')
Let’s see the image

14
 We see the resulting binarized image, next to the original to compare it. We see how the
optimal thresh is spotted by a red line in the histogram of the image.
If the image doesn't have high contrast or the background is uneven, local thresholding
produces better results. Import threshold_local(), also from filters. With this function, we
calculate thresholds in small pixel regions surrounding each pixel we are binarizing. So we
need to specify a block_size to surround each pixel; also known as local neighborhoods. And
an optional offset, that's a constant subtracted from the mean of blocks to calculate the local
threshold value. Here in the threshold_local function we set a block_size of 35 pixels and an
offset of 10. Then apply that local thresh.
Code:
# Import the local threshold function
from skimage.filters import threshold_local
# Set the block size to 35
block_size = 35
# Obtain the optimal local thresholding
local_thresh = threshold_local(text_image, block_size, offset=10)
# Apply local thresholding and obtain the binary image
binary_local = text_image > local_thresh
# Show the original and binarized image
show_image(image, 'Original')
show_image(binary_local, 'Local thresholding')
Showing the original and the resulting image, we can see it works well in this image that
has different lighting conditions.

15
EXERCISE:
Exercise 1:
Question: Whats the main difference between the images shown below?

These images have been preloaded as coffee_image and coins_image from the
scikit-image data module using:

coffee_image = data.coffee()

coins_image = data.coins()

Choose the right answer that best describes the main difference related to color and
dimensional structure.
In the console, use the function shape() from NumPy, to obtain the image shape
(Height, Width, Dimensions) and find out. NumPy is already imported as np.

Code
# Import the modules from skimage
from skimage import ____, ____
Instruction
 Both have 3 channels for RGB-3 color representation
 coffee_image has a shape of (303, 384), grayscale. And coins_image (400, 600, 3),
RGB-3
16
  coins_image has a shape of (303, 384), grayscale. And coffee_image (400, 600,
3), RGB-3
 Both are grayscale, with single color dimension

Exercise 2: RGB to Grayscale

Question:
In this exercise you will load an image from scikit-image module data and make it
grayscale, then compare both of them in the output.
We have preloaded a function show_image(image, title='Image') that
displays the image using Matplotlib.
You can check more about its parameters using ?show_image() or
help(show_image) in the console.

Code

# Import the modules from skimage

from skimage import ____, ____
# Load the rocket image
rocket = data.____()
# Convert the image to grayscale
gray_scaled_rocket = color.____(____)
# Show the original image
show_image(rocket, 'Original RGB image')
17
 # Show the grayscale image
show_image(gray_scaled_rocket, 'Grayscale image')
Instruction

 Import the data and color modules from Scikit image. The first module provides
example images, and the second, color transformation functions.
 Load the rocket image.
 Convert the RGB-3 rocket image to grayscale.

Exercise 3: Flipping out

Question

As a prank, someone has turned an image from a photo album of a trip to Seville upside-
down and back-to-front! Now, we need to straighten the image, by flipping it.

Image loaded as flipped_seville.

Using the NumPy methods learned in the course, flip the image horizontally and
vertically. Then display the corrected image using the show_image() function.
NumPy is already imported as np.

Code

# Flip the image vertically

seville_vertical_flip = ____.____(flipped_seville)
18
Instruction: Flip the image vertically

Exercise 4: Histograms
Question:

In this exercise, you will analyze the amount of red in the image. To do this, the
histogram of the red channel will be computed for the image shown below:

Image loaded as image

Extracting information from images is a fundamental part of image enhancement. This

way you can balance the red and blue to make the image look colder or warmer.
You will use hist() to display the 256 different intensities of the red color.
And ravel() to make these color values an array of one flat dimension.
Matplotlib is preloaded as plt and Numpy as np.
Remember that if we want to obtain the green color of an image we would do the
following:

green = image[:, :, 1]

19
Code

# Obtain the red channel

red_channel = image[____, ____, ____]
# Plot the red histogram with bins in a range of 256
plt.____(____.____, bins=____)
# Set title and show
plt.title('Red Histogram')
plt.show()
Instruction

 Obtain the red channel using slicing.

 Plot the histogram and bins in a range of 256. Don't forget .ravel() for the color
channel

Exercise 5: Apply global thresholding

Question

In this exercise, you'll transform a photograph to binary so you can separate the
foreground from the background.
To do so, you need to import the required modules, load the image, obtain the optimal
thresh value using threshold_otsu() and apply it to the image.
You'll see the resulting binarized image when using the show_image() function,
previously explained.

20
 Image loaded as chess_pieces_image.

Remember we have to turn colored images to grayscale. For that we will use the
rgb2gray() function learned in previous video. Which has already been imported for you.

Code

# Import the otsu threshold function

from skimage.____ import ____
# Make the image grayscale using rgb2gray
chess_pieces_image_gray = ____(____)
# Obtain the optimal threshold value with otsu
thresh = ____(____)
# Apply thresholding to the image
binary = chess_pieces_image_gray ____ ____
# Show the image
show_image(binary, 'Binary image')
Exercise 6: When the background isn't that obvious
Question

Sometimes, it isn't that obvious to identify the background. If the image background is
relatively uniform, then you can use a global threshold value as we practiced before,
using threshold_otsu(). However, if there's uneven background illumination, adaptive
thresholding threshold_local() (a.k.a. local thresholding) may produce better results.

21
 In this exercise, you will compare both types of thresholding methods (global and local),
to find the optimal way to obtain the binary image we need.

Image loaded as page_image.

Code

# Import the otsu threshold function

from skimage.____ import ____
# Obtain the optimal otsu global thresh value
global_thresh = ____(page_image)
# Obtain the binary image by applying global thresholding
binary_global = page_image ____ ____
# Show the binary image obtained
show_image(binary_global, 'Global thresholding')
Instruction

 Import the otsu threshold function, obtain the optimal global thresh value of the image,
and apply global thresholding
 Import the local threshold function, set block size to 35, obtain the local thresh value,
and apply local thresholding

Exercise 7: Trying other methods

Question

As we saw in the video, not being sure about what thresholding method to use isn't a
problem. In fact, scikit-image provides us with a function to check multiple methods and see
for ourselves what the best option is. It returns a figure comparing the outputs of
different global thresholding methods.

22
 Image loaded as fruits_image.

You will apply this function to this image, matplotlib.pyplot has been loaded
as plt. Remember that you can use try_all_threshold() to try multiple global
algorithms.

Code

# Import the try all function

from skimage.____ import ____
# Import the rgb to gray convertor function
from skimage.____ import ____
# Turn the fruits_image to grayscale
grayscale = ____
# Use the try all method on the resulting grayscale image
fig, ax = ____(____, verbose=False)
# Show the resulting plots
plt.show()
Instruction

 Import the try all function

 Import the rgb to gray convertor function
 Turn the fruits image to grayscale
 Use the try all method on the resulting grayscale image

Exercise 8: Apply thresholding

Question:

23
 In this exercise, you will decide what type of thresholding is best used to binarize an
image of knitting and craft tools. In doing so, you will be able to see the shapes of the objects,
from paper hearts to scissors more clearly.

Image loaded as tools_image

What type of thresholding would you use judging by the characteristics of the image? Is
the background illumination and intensity even or uneven?

Code

# Import threshold and gray convertor functions

from skimage.____ import ____
from skimage.color import ____
# Turn the image grayscale
gray_tools_image = ____
# Obtain the optimal thresh
thresh = ____(gray_tools_image)
# Obtain the binary image by applying thresholding
binary_image = gray_tools_image > ____
# Show the resulting binary image
show_image(binary_image, 'Binarized image')

24
 CHAPTER 2. FILTERS
THEORIES:
In this new chapter we'll learn about filtering, contrast, transformations and morphology

Filtering
1.4. Filters
Filtering is a technique for modifying or enhancing an image. In essence, a filter is a
mathematical function that is applied to images. It can be used to emphasize or remove certain
features, like edges. Smoothing, sharpening and edge detection. In this video we will cover
smoothing and edge detection. Filtering is a neighborhood operation, lets see what that means.

1.4.1. Neighborhoods

Certain image processing operations involve processing an image in sections, called

blocks or neighborhoods, rather than processing the entire image at once. This is the case for
filtering, histogram equalization for contrast enhancement, and morphological functions, all
three of which use this approach.

1.4.2. Edge detection

So, with filtering we can detect edges. This technique can be used to find the boundaries
of objects within images. As well as segment and extract information like how many coins are
in an image. Most of the shape information of an image is enclosed in edges.

25
 Edge detection works by detecting discontinuities in brightness. Like in this image, where
we spot the chocolate kisses shapes in the image.

A common edge detection algorithm is Sobel. This is a filter that we can find in scikit
image's module filters with the sobel function. The previously used coins image is preloaded as
image coins. We apply the filter by passing the image we want to detect the edges from as
parameter. This function requires a 2-dimensional grayscale image as input. So in the case of a
colored image, we'll need to convert it to grayscale first. Then, we show the original and the
resulting image with a function that uses Matplotlib subplots.
So we see that the filter is detecting the edges in the original image and highlighting the
boundaries as closed lines in the resulting image.
Code:

# Import module and function

from skimage.filters import sobel
# Apply edge detection filter
edge_sobel = sobel(image_coins)
# Show original and resulting image to compare
plot_comparison(image_coins, edge_sobel, "Edge with Sobel")

26
1.4.3. Comparing plots

This is the plot comparison function we just used. It plots the original image next to the
resulting one. Using this function allows us to reuse this code and focus more on the image
processing part of the code.

def plot_comparison(original, filtered, title_filtered):

fig, (ax1, ax2) = plt.subplots(ncols=2, figsize=(8, 6),
sharex=True,
sharey=True)
ax1.imshow(original, cmap=plt.cm.gray)
ax1.set_title('original')
ax1.axis('off')
ax2.imshow(filtered, cmap=plt.cm.gray)
ax2.set_title(title_filtered)
ax2.axis('off')

1.4.4. Gaussian smoothing

Lets look at another filtering technique, smoothing. We can achieve this with a Gaussian
filter. This technique is typically used to blur an image or to reduce noise. Here we see the
effect of applying a Gaussian filter, which can especially be seen in the edges of the dogs hair.
The Gaussian filter will blur edges and reduce contrast. This is used in other techniques like
anti-aliasing filtering.

Code:

# Import the module and function

from skimage.filters import gaussian
27
 # Apply edge detection filter
gaussian_image = gaussian(amsterdam_pic, multichannel=True)
# Show original and resulting image to compare
plot_comparison(amsterdam_pic, gaussian_image, "Blurred with
Gaussian filter")

Lets see how we can do this with Scikit image using this picture in Amsterdam.

We import the Gaussian function from the filters module of scikit-image. To apply the
filter, the original image is passed as first parameter to the Gaussian function and the
multichannel boolean parameter is set to True if the image is colored, otherwise it needs to be
set to False. Finally, lets compare the original and the resulting image.

1.5. Contrast
1.5.1. Contrast enhancement
28
 Image enhancement can be extremely useful in multiple areas. Often medical images like
this X-ray can have low contrast, making it hard to spot important details. When we improve
the contrast,the details become more visible. It's definitely easier to spot things on this one!

1.5.2. Contrast

The contrast of an image can be seen as the measure of its dynamic range, or the "spread"
of its histogram. Consider this image. The contrast is the difference between the maximum and
minimum pixel intensity in the image. The histogram of this image is shown on the right. The
maximum value of pixel intensity is 255 while the minimum is 0. 255 - 0 = 255.

An image of low contrast has small difference between its dark and light pixel values. Is
usually skewed either to the right (being mostly light), to the left (when is mostly dark), or
located around the middle (mostly gray).

29
1.5.3. Enhance contrast

We can enhance contrast through contrast stretching which is used to stretch the
histogram so the full range of intensity values of the image is filled. And histogram
equalization, that spreads out the most frequent histogram intensity values using probability
distribution. We'll cover histogram equalization in this video.
In general, there are three types of histogram equalization. The standard, the adaptive, and
the limited adaptive. In scikit-image we can apply standard histogram equalization, contrast
stretching, and contrast limited adaptive as we can see in these images.

1.5.4. Histogram equalization

As we have seen Histogram equalization spreads out the most frequent intensity values.

To apply this type of histogram equalization import the exposure module from scikit-
image. We then have access to all equalization methods. In this case, the equalize_hist
function, applies normal histogram equalization to the original image.
Code:

from skimage import exposure

# Obtain the equalized image
image_eq = exposure.equalize_hist(image)
# Show original and result
show_image(image, 'Original')
show_image(image_eq, 'Histogram equalized')
30
 Then we show both images, the original and equalized one to see the difference.

We get a result that, despite the increased contrast, doesn't look natural. In fact, it doesn't
even look like the image has been enhanced at all.

1.5.5. Adaptive Equalization

Another type of histogram equalization is the adaptive one. This method computes
several histograms, each corresponding to a distinct part of the image, and uses them to
redistribute the lightness values of the image histogram. A type of this method is the
Contrastive Limited Adaptive Histogram Equalization (CLAHE) which was developed to
prevent over-amplification of noise that adaptive histogram equalization can give rise to. In this
image, we see the result of the CLAHE method and it may seem very similar to the standard
method.

1.5.6. Contrastive Limited Adaptive Equalization

But if you look closer and compare the results, you will see that the adaptive method is
not that intense, so it looks more natural. This is because it is not taking the global histogram of
the entire image, but operates on small regions called tiles or neighborhoods.

31
1.5.7. CLAHE in scikit-image

To apply this kind of adaptive equalization we can use the equalize_adapthist function
provided by scikit-image. It calculates the contrast transform function for each tile individually.
We pass the original image as first parameter and a clip_limit. This clipping limit, is
normalized between 0 and 1 (higher values give more contrast).
Code:

from skimage import exposure

# Apply adaptive Equalization
image_adapteq = exposure.equalize_adapthist(image,
clip_limit=0.03)
# Show original and result
show_image(image, 'Original')
show_image(image_adapteq, 'Adaptive equalized')

Then, show the original and resulting images.

Comparing them, the resulting image is enhanced and we can better detail small objects
and figures. Like the footprints in the ground.

1.6. Transformations
1.6.1. Why transform images?
32
 For example when you need to pass images to a Machine Learning model so it can
classify if it's a cat or a dog, and we want the image to be upright. To optimize the size of
images so it doesn't take long to analyze them. Or when we need all the images to have the
same proportion before processing them further.

1.6.2. Rotating

We have flipped images before using Numpy. Rotating images allows us to apply angles,
like when you rotate an image 90 degrees, clockwise. Meaning, to the right. Or like when you
rotate images 90 degrees anticlockwise, meaning, to the left. Like we see in this example
image.

Rotating clockwise: We can use the rotate function from scikit-image module
"transform" to rotate an image by a certain angle around its center Once we import the module
and function. In this code we are obtaining the rotated image with 90 degrees clockwise. The
first parameter is the image we want to rotate and the second parameter is the angle. The
rotation angle is in degrees in counter-clockwise or anticlockwise direction. So we use negative
values. Here, you can see the original and rotated images.
Code:
from skimage.transform import rotate
# Rotate the image 90 degrees clockwise
image_rotated = rotate(image, -90)
show_image(image_rotated, 'Original')

33
 show_image(image_rotated, 'Rotated 90 degrees clockwise')
Rotating anticlockwise: If we want to rotate anticlockwise, that is, to the left, we need to
specify the degrees angles with positive numbers. In this case, we are rotating 90 degrees to the
left.
Code:
from skimage.transform import rotate
# Rotate the image 90 degrees clockwise
image_rotated = rotate(image, 90)
show_image(image_rotated, 'Original')
show_image(image_rotated, 'Rotated 90 degrees anticlockwise')

1.6.3. Rescaling

The rescale operation resizes an image by a given scaling factor. This can either be a
single floating point value, or multiple values - one for each axis.

We can use the rescale function from the transform module. Once imported, in this
example we make an image to be 4 times smaller than its original size by setting the scaling
factor to 1/4. Setting an anti_aliasing boolean parameter to true specifies if applying a filter to
smooth the image prior to down-scaling. We'll get into more detail later on. Then a
multichannel if the image is colored.
Code: (Downgrading)
from skimage.transform import rescale
# Rescale the image to be 4 times smaller
image_rescaled = rescale(image, 1/4, anti_aliasing=True,
multichannel=True)
show_image(image, 'Original image')
34
 show_image(image_rescaled, 'Rescaled image')

1.6.4. Aliasing in digital images

In a digital image, aliasing is a

pattern or a rippling effect. Aliasing
makes the image look like it has
waves or ripples radiating from a
certain portion. This happens because
the pixelation of the image is poor;
when they simply do not look right.
Here, we applied a resizing of 1/30, and we see what the anti_aliasing filter is doing to the
image when is set. The first one has the anti_aliasing to True so we see is softer. While the one
without it is pixelated.

1.6.5. Resizing

Resizing is used for making images match a certain size. The same purpose as rescale, but
allows to specify an output image shape instead of a scaling factor.

35
 We can use the resize function from the transform module. This function requires the
original image as the first parameter and a tuple holding the desired height and width to resize
the image. Optionally, we can also set the anti-aliasing method.
Code:
from skimage.transform import resize
# Height and width to resize
height = 400
width = 500
# Resize image
image_resized = resize(image, (height, width),
anti_aliasing=True)
# Show the original and resulting images
show_image(image, 'Original image')
show_image(image_resized, 'Resized image')
We can see how the image has been resized to a height of 400 and a width of 600.

Resizing proportionally: We obtain a good looking and proportionally accurate resizing.

Code:
from skimage.transform import resize
# Set proportional height so its 4 times its size
height = image.shape[0] / 4
width = image.shape[1] / 4
# Resize image
image_resized = resize(image, (height, width),
anti_aliasing=True)
show_image(image_resized, 'Resized image')

36
1.7. Morphology:
When you try to spot objects in an image, you can do so by its characteristics, like the
shape. This what Morphology does

1.7.1. Binary images

Binary regions
produced by simple
thresholding can be distorted
by noise and texture, as we
can see in the image.

1.7.2. Morphological filtering

Morphological filtering operations try to remove

these imperfections by accounting for the form and
structure of the objects in the image. These operations
are especially suited to binary images, but some can
extend to grayscale ones.

1.7.3. Morphological operations

Basic morphological operations are dilation and erosion. Dilation adds pixels to the
boundaries of objects in an image, while erosion removes pixels on object boundaries. The
number of pixels added or removed from the objects in an image depends on the size and shape
of a structuring element used to process the image.

37
1.7.4. Structuring element

The structuring element is a small binary image used to probe the input image. We try to
"fit" in the image object we want to get its shape. Here, the square structuring element named
"A" fits in the object we want to select. The "B" intersects the object and the "C" is out of it. So
if we want to select an apple in a table, we want the structuring element fit in that apple so then
expands, probe and obtain the shape.

The dimensions specify the size of the structuring element. Like a square of 5 by 5 pixels.
The pattern of ones and zeros specifies the shape of the structuring element. This should be of a
similar form to the shape of the object we want to select. So we see in here different types of
shapes, from squares, to diamond. The pink cell is the center or origin of the structuring
element. Identifies the pixel being processed.

1.7.5. Shapes in scikit-image

scikit-image has multiple shapes for this structured element, each one with its own
method from the morphology module. If we want square as the structured element, we can

38
obtain it with the square method. Or a rectangle with width and height. This will return the
desired shape and if we print we'll see how these are formed with 1s.

from skimage import morphology rectangle =

square = morphology.square(4) morphology.rectangle(4, 2)

[[1 1 1 1]
[1 1 1 1] [[1 1]
[1 1 1 1] [1 1]
[1 1 1 1]] [1 1]
[1 1]]

1.7.6. Erosion in scikit-image

To apply erosion we can use the binary erosion function. With this we can optionally set a
structuring element to use in the operation. Here we import it and load a binary horse image.
Set the structuring element to a rectangular-shaped, since it's somewhat similar to the shape we
want to obtain, which is a horse. And obtain the eroded image by using this function, passing
the image and structuring element as parameters. If not set, the function will use a cross-shaped
structured element.
Code:

from skimage import morphology

# Set structuring element to the rectangular-shaped
selem = rectangle(12,6)
# Obtain the erosed image with binary erosion
eroded_image = morphology.binary_erosion(image_horse,
selem=selem)
# Show result
plot_comparison(image_horse, eroded_image, 'Erosion')
Showing the resulting image, next to the original to compare them, we see that the
resulted image is missing some pixels. But still kind of showing the horse shape.

39
 Binary erosion with default selem: If we apply binary erosion with default structuring
element shape, we would obtain this eroded image. It's working better, more accurate than
before. So for this image the cross-shaped works great.
Code:

# Binary erosion with default selem

eroded_image = morphology.binary_erosion(image_horse)

1.7.7. Dilation in scikit-image

Now let's look at dilation. As the name implies, this operation sort of "expands" the
objects in the image. Here, we use binary dilation function, also from the morphology module,
on image_horse. Let's use the default structuring element, which is cross-shaped.
Code:
from skimage import morphology
# Obtain dilated image, using binary dilation
dilated_image = morphology.binary_dilation(image_horse)
# See results
plot_comparison(image_horse, dilated_image, 'Erosion')

40
EXERCISE:
Exercise 1: Edge detection
Question: Detect edges in an image by applying the Sobel filter

Image preloaded as soaps_image.

The show_image() function has been already loaded for you

Code

# Import the color module

from ____ import ____
# Import the filters module and sobel function
from skimage.____ import ____
# Make the image grayscale
soaps_image_gray = ____.____(soaps_image)
# Apply edge detection filter
edge_sobel = ____(____)
# Show original and resulting image to compare
show_image(soaps_image, "Original")
show_image(edge_sobel, "Edges with Sobel")

Instruction

 Import the color module so you can convert the image to grayscale

41
  Import the sobel() function from filters module
 Make soaps_image grayscale using the appropriate method from the color
module
 Apply the sobel edge detection filter on the obtained image grayscale
soaps_image_gray

Exercise 2: Blurring to reduce noise

Question: In this exercise you will reduce the sharpness of an image of a building taken
during a London trip, through filtering.

Image loaded as building_image

Code:

# Import Gaussian filter

____

# Apply filter

gaussian_image = ____

42
# Show original and resulting image to compare

show_image(____, "Original")

____(____, "Reduced sharpness Gaussian")

Instruction:

 Import the Gaussian filter

 Apply the filter to the building_image, set the multichannel parameter to the
correct value
 Show the original building_image and resulting gaussian_image

Exercise 3:
Question: What is the contrast of this image?

Instruction:

 The histogram tell us

 Just as we saw previously, you can calculate the contrast by calculating the range of the
pixel intensities i.e. by subtracting the minimum pixel intensity value from
the histogram to the maximum one
 You can obtain the maximum pixel intensity of the image by using
the np.max() method from NumPy and the minimum with np.min() in the
console

 The image has already been loaded as clock_image, NumPy as np and

the show_image() function

 Possible Answers

43
  The contrast is 255 (high contrast)
 The contrast is 148
 The contrast is 189
 The contrast is 49 (low contrast)
Exercise 4: Medical images
Question: You are trying to improve the tools of a hospital by pre-processing the X-ray
images so that doctors have a higher chance of spotting relevant details. You'll test our code on
a chest X-ray image from the National Institutes of Health Chest X-Ray Dataset

Image loaded as chest_xray_image.

First, you'll check the histogram of the image and then apply standard histogram
equalization to improve the contrast. Remember we obtain the histogram by using
the hist() function from Matplotlib, which has been already imported as plt
Code:

# Import the required module

from ____ import ____

Instruction: Import the required Scikit-image module for contrast

44
Exercise 5: Aerial image
Question: In this exercise, we will improve the quality of an aerial image of a city. The
image has low contrast and therefore we can not distinguish all the elements in it.

Image loaded as image_aerial.

For this we will use the normal or standard technique of Histogram Equalization
Code:

# Import the required module

from ____ import ____
# Use histogram equalization to improve the contrast
image_eq = ____
# Show the original and resulting image
show_image(image_aerial, 'Original')
show_image(____, 'Resulting image')

Instruction:

 Import the required module from scikit-image

 Use the histogram equalization function from the module previously imported
 Show the resulting image

45
Exercise 6: Let's add some impact and contrast
Question: Have you ever wanted to enhance the contrast of your photos so that they appear
more dramatic?

In this exercise, you'll increase the contrast of a cup of coffee. Something you could share
with your friends on social media. Don't forget to use #ImageProcessingDatacamp as
hashtag!

Even though this is not our Sunday morning coffee cup, you can still apply the same
methods to any of our photos.

A function called show_image(), that displays an image using Matplotlib, has already
been defined. It has the arguments image and title, with title being 'Original' by
default.

Code:

# Import the necessary modules

from skimage import data, ____
# Load the image
original_image = ____.coffee()
# Apply the adaptive equalization on the original image
adapthist_eq_image = ____.____(original_image, ____=____)
# Compare the original image to the equalized
show_image(original_image)
show_image(adapthist_eq_image, '#ImageProcessingDatacamp')

Instruction:

46
  Import the module that includes the Contrast Limited Adaptive Histogram Equalization
(CLAHE) function
 Obtain the image you'll work on, with a cup of coffee in it, from the module that holds all
the images for testing purposes
 From the previously imported module, call the function to apply the adaptive equalization
method on the original image and set the clip limit to 0.03

Exercise 7: Aliasing, rotating and rescaling

Question: Let's look at the impact of aliasing on images.

Remember that aliasing is an effect that causes different signals, in this case pixels, to
become indistinguishable or distorted.

You'll make this cat image upright by rotating it 90 degrees and then rescaling it two
times. Once with the anti aliasing filter applied before rescaling and a second time without it,
so you can compare them.

Image preloaded as image_cat.

Code:

# Import the module and the rotate and rescale functions

47
 from skimage.____ import ____, ____

Instruction: Import the module and the rotating and rescaling functions

Exercise 8: Enlarging images

Question:

Have you ever tried resizing an image to make it larger? This usually results in loss of
quality, with the enlarged image looking blurry.

The good news is that the algorithm used by scikit-image works very well for enlarging
images up to a certain point. In this exercise you'll enlarge an image three times!

You'll do this by rescaling the image of a rocket, that will be loaded from the data module

Code:

# Import the module and function to enlarge images

from skimage.____ import ____
# Import the data module
from skimage import ____
# Load the image from data
rocket_image = ____.____()
# Enlarge the image so it is 3 times bigger
enlarged_rocket_image = ____(rocket_image, ____, ____=____, multicha
nnel=____)
48
 # Show original and resulting image
show_image(rocket_image)
show_image(enlarged_rocket_image, "3 times enlarged image")

Introduction:
 Import the module and function needed to enlarge images, you'll do this by rescaling.
 Import the data module.
 Load the rocket() image from data.
 Enlarge the rocket_image so it is 3 times bigger, with the anti aliasing filter
applied. Make sure to set multichannel to True or you risk your session timing
out!

Exercise 9: Proportionally resizing

Question: We want to downscale the images of a veterinary blog website so all of them
have the same compressed size. It's important that you do this proportionally, meaning that
these are not distorted. First, you'll try it out for one image so you know what code to test later
in the rest of the pictures.

The image preloaded as dogs_banner.

Remember that by looking at the shape of the image, you can know its width and height.

Code:

# Import the module and function

49
 from skimage.___ import ____
# Set proportional height so its half its size
height = int(____ / 2)
width = int(____ / 2)
# Resize using the calculated proportional height and width
image_resized = ____(dogs_banner, (____, ____), anti_aliasing=True)
# Show the original and rotated image
show_image(dogs_banner, 'Original')
show_image(image_resized, 'Resized image')

Instruction:

 Import the module and function to resize.

 Set the proportional height and width so it is half the image's height size.
 Resize using the calculated proportional height and width.

Exercise 10: Morphology

Question: Handwritten letters

A very interesting use of computer vision in real-life solutions is performing Optical

Character Recognition (OCR) to distinguish printed or handwritten text
characters inside digital images of physical documents. Let's try to
improve the definition of this handwritten letter so that it's easier to
classify.

As we can see it's the letter R, already binary, with some noise in it. It's already loaded
as upper_r_image. Apply the morphological operation that will discard the pixels near the
letter boundaries.

Code:

# Import the morphology module

from ____
# Obtain the eroded shape
eroded_image_shape = ____.____(____)
# See results
show_image(upper_r_image, 'Original')
50
 show_image(eroded_image_shape, 'Eroded image')

Instruction:

 Import the module from scikit-image.

 Apply the morphological operation for eroding away the boundaries of regions of
foreground pixels.

Exercise 11: Improving thresholded image

Question: In this exercise, we'll try to reduce the noise of a thresholded image using the
dilation morphological operation.

Image already loaded as world_image.

This operation, in a way, expands the objects in the image.

Code:

# Import the module

from skimage import ____
# Obtain the dilated image
dilated_image = ____
# See results
show_image(world_image, 'Original')
show_image(dilated_image, 'Dilated image')

Instruction:

51
 Import the module

 Obtain the binarized and dilated image, from the original image world_image

52
 CHAPTER 3. IMAGE RESTORATION
THEORIES
3.1. Image restoration
1.7.8. Restore an image

Have you ever wondered if it's possible to restore a damaged or defect image? This could
have happened because your laptop memory got corrupted. Or, it could also be a picture of
your grandparents that over time has been scratched and now is somewhat deteriorated.
Whatever happened, the cool thing is that yes, you can restore them.

1.7.9. Image reconstruction

Besides fixing damaged images,

image restoration or reconstruction, is also
used for text removing, deleting logos
from images and even removing small
objects, like tattoos you prefer not to show
on a picture.
Reconstructing lost or deteriorated parts of images is known as inpainting. The
reconstruction is supposed to be performed in a fully automatic way by exploiting the
information presented in non-damaged regions of the image.

53
 In scikit-image, we can apply inpainting with the inpaint biharmonic function, from the
restoration module. It needs the location of the damaged pixels to be filled, as a mask image on
top of the image to work with. A mask image is simply an image where some of the pixel
intensity values are zero, and others are non-zero.

Image reconstruction in scikit-image: To use this function we first need to import the
module and function. Having the image with the defect already loaded, we need to set the mask
with the location of the damaged pixels in the image. Remember that images are matrix of
pixels. For this, in the course, we use the get_mask() function. This is something that we'll take
a closer look at in a moment. Once we have the mask, we can invoke the inpaint_biharmonic
function to apply it to the image with the defect. This will be passed as the first parameter, then
the mask, and optionally we can set a boolean for the multichannel. Set it to true if the image is
colored.
Code:
from skimage.restoration import inpaint
# Obtain the mask
mask = get_mask(defect_image)
# Apply inpainting to the damaged image using the mask
restored_image = inpaint.inpaint_biharmonic(defect_image,
mask,
multichann
el=True)
# Show the resulting image
show_image(restored_image)
# Show the defect and resulting images
show_image(defect_image, 'Image to restore')
show_image(restored_image, 'Image restored')
54
 Then we can look at the resulting image.

We look at the original image with the defect and the resulting restored image to compare
them. In this example, we can see how the masked pixels get inpainted by the inpainting
algorithm based on the biharmonic equation assumption.

1.7.10.Masks

Imagine you have an old picture of your parents you want to fix. In this image, we
intentionally added the missing pixels by setting them to black. In case you want to remove an
object you can manually delineate it in the mask. And if you want to automatically detect it,
you would need to use Thresholding or segmentation to do so. Something we will learn later
on. In the right image we see the damaged areas of the image as a mask.

The scikit-image inpainting function requires the mask to be an array of pixels that need
to be inpainted. This mask has to be the same shape as one of the image channels. Unknown
pixels have to be represented with 1 and known pixels with 0. So we add the missing pixels by
copying the image and turning the pixels into a numpy array of zeros, meaning it's empty. We
only copy the width and height dimensions of the image, excluding the color dimension, in this
case RGB-3. And then, set the 1s in the specific areas we want to be treated as lost.
Code:
def get_mask(image):

55
 ''' Creates mask with three defect regions '''
mask = np.zeros(image.shape[:-1])
mask[101:106, 0:240] = 1
mask[152:154, 0:60] = 1
mask[153:155, 60:100] = 1
mask[154:156, 100:120] = 1
mask[155:156, 120:140] = 1
mask[212:217, 0:150] = 1
mask[217:222, 150:256] = 1
return mask

3.2. Noise
1.7.11.Noise

How can an image have noise? Well images are signals and real-world signals usually
contain departures from the ideal signal, which is the perfect image, as we observe with our
eyes in real life. Such departures are referred to as noise. We can see how this image has some
color grains when zoomed in.

More specifically, noise is the result of errors in the image acquisition process that result
in pixel values that do not reflect the true intensities of the real scene. In this image we can see
how there is a variation of brightness and color that does not correspond to reality, which is
produced by the camera.

56
1.7.12.Apply noise in scikit-image:

We can add noise to an image using scikit-image, with the random noise function from
the "util" module. The image of a dog has already been preloaded. Now, we need to import the
util module and the random_noise function. We can then add random noise of various types to
the image by using this function. After doing this, we obtain a new image named noisy_image.
To compare the original and resulting noisy image we can show them side-by-side using the
show_image function we introduced in previous videos.
Code:
# Import the module and function
from skimage.util import random_noise
# Add noise to the image
noisy_image = random_noise(dog_image)
# Show original and resulting image
show_image(dog_image)
show_image(noisy_image, 'Noisy image')
By using the random_noise function, we obtain the original image with a lot of added
noise, that is distributed randomly. This type of noise is known as "salt and pepper" because
that's in fact what it looks like. So we now know how to "mess up" a picture and use it for
testing purposes.

57
1.7.13.Reducing noise

Most of the times we will want to remove or reduce the noise of images instead of adding
it in. Like in this example, you can see a noisy image of me that is being denoised. For that, we
can use several algorithms in scikit-image. The higher the resolution of the image, the longer it
may take to eliminate the noise.

Denoising types: Some types of denoising algorithms are: The total variation filter. This
filter tries to minimize the total variation of the image. It tends to produce “cartoon-like”
images, that is, piecewise-constant images. Bilateral filtering smooths images while preserving
edges. It replaces the intensity of each pixel with a weighted average of intensity values from
nearby pixels. In this course, we'll focus on these two. But there are also The wavelet denoising
filter and Non-local means denoising.
We can perform total variation filter denoising with the denoise tv chambolle function
from the restoration module. When applying this function, we can optionally set the denoising
weight. The greater the weight, the more denoising but it could also make the image smoother.
We can also specify if the image is multichannel (colored) or not. The final step is to show the
original image compared with the resulting one.
58
 Code
from skimage.restoration import denoise_tv_chambolle
# Apply total variation filter denoising
denoised_image = denoise_tv_chambolle(noisy_image,
weight= 0.1,
multichannel= Tru
e)
# Show denoised image
show_image(noisy_image, 'Noisy image')
show_image(denoised_image, 'Denoised image')
We see that we obtain a denoised image with the edges preserved but a little bit smooth or
blurry

Bilateral filter denoising is accessible through the denoise bilateral function also from
the restoration module. We apply it to the noisy image, set multichannel to the appropriate
value, and we can leave other options to the default. Now we'll see the original and the
resulting one to compare.
Code
from skimage.restoration import denoise_bilateral
# Apply bilateral filter denoising
denoised_image = denoise_bilateral(noisy_image,
multichannel=True)
# Show original and resulting images
show_image(noisy_image, 'Noisy image')

59
 show_image(denoised_image, 'Denoised image')
The resulting image is less smooth than the one from the total variation filter. And
preserves the edges a lot more.

3.3. Superpixels & segmentation

1.7.14.Segmentation

The goal is to partition images into regions, or segments, to simplify and/or change the
representation into something more meaningful and easier to analyze.

For example, before a tumor is analyzed in a computed tomography, it has to be detected

and somehow isolated from the rest of the image. Or before recognizing a face, it has to also be
picked out from its background. Previously we learned about Thresholding, which is the
simplest method of segmentation. Separating foreground from background. Now we'll learn
about separating more than that.

60
1.7.15.Image representation

Consider how we represent images. They are represented as a grid of pixels. The issue is
that they're not a completely natural representation of an image. If I were to take a single pixel
from the image on the left and then showed it to you on the right, would you be able to tell that
the pixel came from a puppy and that this single pixel holds a logical meaning? A single pixel,
standing alone by itself, is not a natural representation.

1.7.16.Superpixels

So, we can explore more logical meanings in an

image that's formed by bigger regions or grouped
pixels. These are known as superpixels. A superpixel
is a group of connected pixels with similar colors or
gray levels. These carry more meaning than their
simple pixel grid counterparts.
Benefits of superpixels: Superpixel
segmentation is dividing an image into superpixels. It has been applied to many
computer vision tasks, like visual tracking and image classification. Some
advantages for using them are that You can compute features on more
meaningful regions. And that you can reduce an image from
thousands of pixels down to some regions for subsequent
algorithms, so you have computational efficiency.

1.7.17.Segmentation

Two types of segmentation are, Supervised, where some prior

knowledge is used to guide the algorithm. Like the kind of
thresholding in which we specify the threshold value ourselves. And

61
unsupervised where no prior knowledge is required. These algorithms try to subdivide images
into meaningful regions automatically. The user may still be able to tweak certain settings to
obtain the desired output. Like the otsu thresholding we used in first chapter

1.7.18.Unsupervised segmentation

Lets focus on a unsupervised segmentation technique based on superpixels, called Simple

Linear Iterative Clustering or SLIC. It segments the image using a machine learning algorithm
called K-Means clustering. It takes in all the pixel values of the image and tries to separate
them into a predefined number of sub-regions.

We can find the algorithm in the segmentation module as the "slic" function. This method
returns the segmented regions, also known as labels. Here we use this function, with default
parameters and obtain the segments. We'll use the label2rgb method from the color module to
return an image where the segments obtained from the slic method will be highlighted, either
with random colors or with the average color of the superpixel segment. In this example we'll
use the average color. So we pass the segments or labels, the image, and set the kind parameter
to average avg.
Code:
# Import the modules
from skimage.segmentation import slic
from skimage.color import label2rgb
# Obtain the segments
segments = segmentation.slic(image)
# Put segments on top of original image to compare
segmented_image = label2rgb(segments, image, kind='avg')
show_image(image)
show_image(segmented_image, "Segmented image")
62
 Finally, we show the resulting segmented image. You can see how local regions with
similar color and texture distributions are part of the same superpixel group.

If we want more segments, let's say 300, we can specify this with an optional parameter,
n_segments. Its default value is 100 segments.
Code:
# Import the modules
from skimage.segmentation import slic
from skimage.color import label2rgb
# Obtain the segments
segments = segmentation.slic(image, n_segments= 300)
# Put segments on top of original image to compare
segmented_image = label2rgb(segments, image, kind='avg')
show_image(image)
show_image(segmented_image, "Segmented image")
We show the resulting segmented image

3.4. Finding contours

1.7.19.Finding contours

We will learn how to find the contours of the objects in an image. A contour is a closed
shape of points or line segments, representing the boundaries of these objects. Once we find the
63
contours, we can do things like identifying the total points in domino tokens, exactly what we
do in this example, where we count 35! So we can measure size, classify shapes or
determining the number of objects in an image.

Total points in domino tokens: 29

Binary images: The input to a contour-finding function should be a binary image, which
we can produce by first applying thresholding or edge detection. In such binary image, the
objects we wish to detect should be white, while the background remains black.

1.7.20.Find contours using scikit-image

First, the image needs to go through some pre-processing steps. We need to turn the
image to grayscale to later apply thresholding. We can do so by using the rgb2gray function
from the color module.
To use the find_contours function, we need the image we want to obtain contours from, to
be binary. Meaning, black and white. Here we apply thresholding to do that, so we get a
thresholded image.
And then, we use the find contours function, that is included in the measure module of
scikit-image. This function finds the contour lines or joins points(pixels) of equal elevation or
brightness in a 2D array above a given level value. We import the module "measure" from
skimage. And from measure we call the function find_contours. Passing the thresholded image
as the first parameter and a constant level value of zero point eight. We will go into more detail
about the constant level value in the next slide. The function returns a list with all contours of
the image. With the coordinates along the contour.

64
 Code:
# Make the image grayscale
image = color.rgb2gray(image)
# Obtain the thresh value
thresh = threshold_otsu(image)
# Apply thresholding
thresholded_image = image > thresh
# Import the measure module
from skimage import measure
# Find contours at a constant value of 0.8
contours = measure.find_contours(thresholded_image, 0.8)

1.7.21.Constant level value

The level value varies between 0 and 1, the closer to 1 the more sensitive the method is to
detecting contours, so more complex contours will be detected. We have to find the value that
best detects the contours we care for.

1.7.22.The steps to spotting contours

To summarize, the steps for finding the contours of an image are: If it's colored, transform
to grayscale. Obtain the optimal thresh value. Apply thresholding and obtain the binary image.
Once we have our binary image, we can call the find_contours() function and set a constant
level value.
Code:

65
 from skimage import measure
from skimage.filters import threshold_otsu
# Make the image grayscale
image = color.rgb2gray(image)
# Obtain the optimal thresh value of the image
thresh = threshold_otsu(image)
# Apply thresholding and obtain binary image
thresholded_image = image > thresh
# Find contours at a constant value of 0.8
contours = measure.find_contours(thresholded_image, 0.8)
Resulting in the image with contours detected:

1.7.23.A contour's shape

After executing these steps we obtain a list of contours. Each contour is an ndarray of
shape (n, 2), consisting of n row and column coordinates along the contour. In this way, a
contour is like an outline formed by multiple points joined together. The bigger the contour, the
more points joined together and the wider the perimeter formed. Here we can see the shapes of
the contours found in the domino's tokens image.
The first two have a shape of 433, so we deduct they are the outer border contour of the
tokens because they are the longest. Meaning, these are the biggest objects, judging by their
shapes.
The ones that have a shape of 401 belong to the inner border of the domino token. Since
the tokens of the original image had a line around them.
The 123 are the dividing line in the middle of the tokens.
And then we see the majority of the contours have a shape of 59. These are the token
dots. If we count them, we obtain a total number of 7, the total number of dots for both tokens.
(433, 2)
(433, 2) --> Outer border
(401, 2)
66
 (401, 2) --> Inner border
(123, 2)
(123, 2) --> Divisory line of tokens
(59, 2)
(59, 2)
(59, 2)
Number of dots: 7
(57, 2)
(57, 2)
(59, 2)
(59, 2) --> Dots
EXERCISE
Exercise 1: restore a damaged image
Question: In this exercise, we'll restore an image that has missing parts in it, using
the inpaint_biharmonic() function.

Loaded as defect_image.

We'll work on an image from the data module, obtained by data.astronaut(). Some of
the pixels have been replaced by 1s using a binary mask, on purpose, to simulate a damaged
image. Replacing pixels with 1s turns them totally black. The defective image is saved as an
array called defect_image.
The mask is a black and white image with patches that have the position of the image bits
that have been corrupted. We can apply the restoration function on these areas. This mask is
preloaded as mask.
Remember that inpainting is the process of reconstructing lost or deteriorated parts of
images and videos.

Code:

# Import the module from restoration

67
 from ____.____ import ____
Instruction:

Import the inpaint function in the restoration module in scikit-image (skimage).

Exercise 2: Removing logos

Question:

As we saw in the video, another use of image restoration is removing objects from an
scene. In this exercise, we'll remove the Datacamp logo from an image.

Image loaded as image_with_logo.

You will create and set the mask to be able to erase the logo by inpainting this area.
Remember that when you want to remove an object from an image you can either
manually delineate that object or run some image analysis algorithm to find it.

Code:

# Initialize the mask

mask = ____(____[:-1])
# Set the pixels where the logo is to 1
mask[210:272, 360:425] = ____
# Apply inpainting to remove the logo
image_logo_removed = inpaint.____(____, ____,multichannel=True)
# Show the original and logo removed images
show_image(image_with_logo, 'Image with logo')
show_image(image_logo_removed, 'Image with logo removed')
Instruction:

 Initialize a mask with the same shape as the image, using np.zeros()
68
  In the mask, set the region that will be inpainted to 1
 Apply inpainting to image_with_logo using the mask

Exercise 3: Noise
Question: In this exercise, we'll practice adding noise to a fruit image.

Image preloaded as fruit_image.

Code:

# Import the module and function

from skimage.____ import ____
# Add noise to the image
noisy_image = ____
# Show original and resulting image
show_image(____, 'Original')
____(____, 'Noisy image')
Instruction:

 Import the util module and the random noise function

 Add noise to the image
 Show the original and resulting image
Exercise 4: Reducing noise
Question: We have a noisy image that we want to improve by removing the noise in it.
69
 Preloaded as noisy_image.

Use total variation filter denoising to accomplish this

Code:

# Import the module and function

from skimage.____ import ____
# Apply total variation filter denoising
denoised_image = ____(____, multichannel=____)
# Show the noisy and denoised images
show_image(____, 'Noisy')
____(____, 'Denoised image')
Instruction:

 Import the denoise_tv_chambolle function from its module

 Apply total variation filter denoising

 Show the original noisy and the resulting denoised image

Exercise 5: Reducing noise while preserving edges

Question: In this exercise, you will reduce the noise in this landscape picture.

70
 Preloaded as landscape_image

Since we prefer to preserve the edges in the image, we'll use the bilateral denoising filter

Code:

# Import bilateral denoising function

____
# Apply bilateral filter denoising
denoised_image = ____(____, multichannel=____)
# Show original and resulting images
show_image(____, 'Noisy image')
show_image(____, 'Denoised image')
Instruction:

 Import the denoise_bilateral function from its module

 Apply bilateral filter denoising
 Show the original noisy and the resulting denoised image

Exercise 6: Number of pixels

Question: Let's calculate the total number of pixels in this image.

71
 Image preloaded as face_image

The total amount of pixel is its resolution. Given by Height×Width

Instruction: Use .shape from NumPy which is preloaded as np, in the console to check
the width and height of the image

 face_image is 191 * 191 = 36,481 pixels

 face_image is 265 * 191 = 50,615 pixels
 face_image is 1265 * 1191 = 1,506,615 pixels
 face_image is 2265 * 2191 = 4,962,615 pixels
Exercise 7: Superpixel segmentation
Question: In this exercise, you will apply unsupervised segmentation to the same image
(face_image), before it's passed to a face detection machine learning model. So you will reduce
this image from 265×191=50,615 pixels down to 400 regions.

The show_image() function has been preloaded for you as well

Code:

# Import the slic function from segmentation module

from skimage.____ import ____
# Import the label2rgb function from color module
from skimage.____ import ____
# Obtain the segmentation with 400 regions
segments = ____(____, ____= ____)
# Put segments on top of original image to compare
segmented_image = ____(____, ____, kind='avg')
72
 # Show the segmented image
show_image(segmented_image, "Segmented image, 400 superpixels")
Instruction:

 Import the slic() function from the segmentation module.

 Import the label2rgb() function from the color module.
 Obtain the segmentation with 400 regions using slic().
 Put segments on top of original image to compare with label2rgb().

Exercise 8: Contouring shapes

Question: In this exercise we'll find the contour of
a horse.

For that we will make use of a binarized image

provided by scikit-image in its data module. Binarized
images are easier to process when finding contours with
this algorithm. Remember that contour finding only
supports 2D image arrays.
Once the contour is detected, we will display it together with the original image. That
way we can check if our analysis was correct!
show_image_contour(image, contours) is a preloaded function that displays the image
with all contours found using Matplotlib.

Code:

# Import the modules

from skimage import ____, ____
# Obtain the horse image
horse_image = ____.horse()
# Find the contours with a constant level value of 0.8
contours = measure.____(____, ____)
# Shows the image with contours found
show_image_contour(horse_image, contours)
Instruction:

 Import the data and the module needed for contouring detection.
 Obtain the horse image shown in the context area.
 Find the contours of the horse image using a constant level value of 0.8.

73
Exercise 9: Find contours of an image that is not binary
Question: Let's work a bit more on how to prepare an image to
be able to find its contours and extract information from it. We'll
process an image of two purple dices loaded as image_dices and
determine what number was rolled for each dice.

In this case, the image is not grayscale or binary yet. This means
we need to perform some image pre-processing steps before looking for the contours. First,
we'll transform the image to a 2D array grayscale image and next apply thresholding. Finally,
the contours are displayed together with the original image.
color, measure and filters modules are already imported so you can use the functions to
find contours and apply thresholding. We also import io module to load the image_dices from
local memory, using imread.

Code:

# Make the image grayscale

image_dices = color.____(image_dices)
Instruction: Transform the image to grayscale using rgb2gray().

Exercise 10: Count the dots in a dice's image

Question: Now we have found the contours, we can extract information from it.

In the previous exercise, we prepared a purple dices image to find its contours:

This time we'll determine what number was rolled for the dice, by counting the dots in the
image.
The contours found in the previous exercise are preloaded as contours.
Create a list with all contour's shapes as shape_contours. You can see all the contours
shapes by calling shape_contours in the console, once you have created it.

74
 Check that most of the contours aren't bigger in size than 50. If you count them, they are
the exact number of dots in the image.
show_image_contour(image, contours) is a preloaded function that displays the image
with all contours found using Matplotlib

Code:

# Create list with the shape of each contour

shape_contours = [cnt.shape[0] for cnt in ____]
# Set 50 as the maximum size of the dots shape
max_dots_shape = ____
# Count dots in contours excluding bigger than dots size
dots_contours = [cnt for cnt in contours if np.shape(cnt)[0] < ____]
# Shows all contours found
show_image_contour(binary, contours)
# Print the dice's number
print("Dice's dots number: {}. ".format(len(____)))
Instruction:

 Make shape_contours be a list with all contour shapes of contours

 Set max_dots_shape to 50
 Set the shape condition of the contours to be the maximum shape size of the
dots max_dots_shape
 Print the dice's number

75
 CHAPTER 4. FINDING THE EDGES WITH CANNY
THEORIES:
3.5. Finding the edges with Canny
1.7.24.Detecting edges

Most of the shape information of an image is enclosed in edges. In this image we see how
the edges hold information about the domino's tokens. Representing an image by its edges has
the advantage that the amount of data is reduced significantly while retaining most of the
image information, like the shapes.

1.7.25.Edge detection

In the previous chapter you have seen how to detect edges using the Sobel filtering
technique. In this video, you'll learn about one of the most used edge detection techniques, the
Canny edge detection. This is widely considered to be the standard edge detection method in
image processing. And produces higher accuracy detecting edges and less execution time
compared with Sobel algorithm.

The detector can be applied with the canny function from the feature module of scikit-
image. This function requires the image to be a 2-dimensional array, meaning, a grayscale
image. So in this example, we convert the image from RGB-3 to grayscale, using the rgb2gray
method from the color module that we already know from previous chapters. Then we apply

76
the canny detector on the coin's image and obtain the resulting image. Let's show it and see
what it looks like.
Code:
from skimage.feature import canny
# Convert image to grayscale
coins = color.rgb2gray(coins)
# Apply Canny detector
canny_edges = canny(coins)
# Show resulted image with edges
show_image(canny_edges, "Edges with Canny")
We see how the edges are highlighted with thick white lines and that some details are
more pronounced than the rest of the image. We can also spot the boundaries and shapes of
coins; by knowing that for each closed circle or ellipse, there's a coin.

1.7.26.Canny edge detector

The first step of this algorithm is to apply a gaussian filter in order to remove noise in the
image. The same gaussian filter we have seen previously in the course with the gaussian
function from the filters module. So, in the canny function you can optionally set the intensity
of this Gaussian filter to be applied in the image, by using the sigma attribute. The lower the
value of this sigma, the less of gaussian filter effect is applied on the image, so it will spot
more edges. On the other hand, if you set a higher value, more noise will be removed and the
result is going to be a less edgy image. The default value of this parameter is 1. In this example
we set it to 0.5, let's see the effect in the image.
Code:
# Apply Canny detector with a sigma of 0.5
canny_edges_0_5 = canny(coins, sigma=0.5)
77
 # Show resulted images with edges
show_image(canny_edges, "Sigma of 1")
show_image(canny_edges_0_5, "Sigma of 0.5")
The resulting image has a lot more edges than the previous one and this is because noise
was removed before continuing with the rest of the steps in the algorithm.

3.6. Right around the corner

1.7.27.Corner detection

Corner detection is an approach used to extract certain types of features and infer the
contents of an image. It's frequently used in motion detection, image registration, video
tracking, panorama stitching, 3D modelling, and object recognition. We saw in the previous
video how to detect edges with the Canny Edge detector, and before that with Sobel, in chapter
2. Edges are a type of feature in images.

1.7.28.Points of interest

78
 Features are the points of interest which provide rich image content information. Points of
interest are points in the image which are invariant to rotation, translation, intensity, and scale
changes. (Basically, robust and reliable). There are different interest points such as corners and
edges. So corner detection is basically detecting (one type of) interest points in an image.

1.7.29.Corners

A corner can be defined as the intersection of two edges. Intuitively, it can also be a
junction of contours. We can see some obvious corners in this checkerboard image. Or in this
building image on the right.

1.7.30.Matching corners

So by detecting corners as interest points, we can match objects from different

perspectives. Like in this image, where we detect the corners of the original image on the left
and then match them in a downscaled image in the right.

79
 Here is another example of corner matching, this time, in a rotated image. We see how
the relevant points. are still being matched.

1.7.31.Harris corner detector

Harris Corner Detector is a corner detection operator that is widely used in computer
vision algorithms. Here, we see an original
image of a building, and on the right we see the
corners detected by the Harris algorithm,
marked in red.

80
 Let's use this image of a Japanese gate to work with the Harris detector. We can access it
by importing the corner_harris function from the feature module of scikit-image. This function
requires grayscale images, so we need to first convert the image from rgb to gray. We can do
this with the rgb2gray function we used previously. This corner_harris function gives us the
Harris measure response image, meaning, the resulting image showing only the possible
corners that were measured.
Code:
from skimage.feature import corner_harris
# Convert image to grayscale
image = rgb2gray(image)
# Apply the Harris corner detector on the image
measure_image = corner_harris(image)
# Show the Harris response image
show_image(measure_image)
So when we show it,

We see how only some black lines are shown. These are the approximated points where
the corners candidates are.
To find the corners in the measure response image, we can use the corner_peaks function.
Which will return the coordinates of the peaks of the possible corners. Optionally, we can
make sure these peak corners are separated by a minimum distance, in pixels, using the
min_distance parameter. Here we're setting the minimum distance between corners to 5 pixels.
In this image, a total of 122 corners were found from the measure response image.
Code:
# Finds the coordinates of the corners
coords = corner_peaks(corner_harris(image), min_distance=5)
81
 print("A total of", len(coords), "corners were detected.")
Corners detected: Here the image is shown with the detected corners marked in red, using
a preloaded function that uses matplotlib and the resulting coordinates to do so, similar to
show_image.
Code:
# Show image with marks in detected corners
show_image_with_detected_corners(image, coords)

Show image with contours: This function uses the coordinates of the corners to plot cross
red marks on it and show it all together.
Code:
def show_image_with_corners(image, coords, title="Corners
detected"):
plt.imshow(image, interpolation='nearest', cmap='gray')
plt.title(title)
plt.plot(coords[:, 1], coords[:, 0], '+r', markersize=15)
plt.axis('off')
plt.show()

3.7. Face detection

1.7.32.Face detection use cases

Several social networks platforms and smart-phones are using

face detection to know if there is someone in a picture and if so,
apply filters, add focus in the face area, or recommend you to tag
friends. You can even automatically blur faces for privacy protection.
Face detection can be useful in other cases as well. Human faces are
82
able to convey many different emotions such as happiness, sadness and many others. That's
why face detection is the key first step before recognizing emotions.

1.7.33.Detecting faces with scikit-image

With scikit-image, we can detect faces using a machine learning classifier, with just a
couple of lines! In this course, we won't cover machine learning concepts in depth, but it's
important to know that we use a cascade of classifiers, which is like multiple classifiers in one.
You can also use it for other things, like cats, objects, or profile faces, from a side view.

To use the face detector, we import the Cascade class from the feature module. This
detection framework needs an xml file, from which the trained data can be read. In this case
we'll use frontal face files that are included in the data module of scikit-image. Then initialize
the detector, using the Cascade class constructor. Great, we now have the detector ready to be
used on images.
Code:
# Import the classifier class
from skimage.feature import Cascade
# Load the trained file from the module root.
trained_file = data.lbp_frontal_face_cascade_filename()
# Initialize the detector cascade.
detector = Cascade(trained_file)

1.7.34.Let's try detecting faces

Let's try our detector on this image of myself.

83
 To apply the detector on images, we need to use the detect_multi_scale method, from the
same cascade class. This method searches for the object, in this case a face. It creates a window
that will be moving through the image until it finds something similar to a human face.

Searching happens on multiple scales. The window will have a minimum size, to spot the
small or far-away faces. And a maximum size to also find the larger faces in the image.

So this method takes the input image as the first parameter, a scale factor, by which the
searching window is multiplied in each step, a step ratio, in which 1 represents an exhaustive
search and usually is slow. By setting this parameter to higher values the results will be worse
but the computation will be much faster. Usually, values in the interval 1 to 1.5 give good
results. Then the minimum and maximum window size are defined. These specify the interval
for the search windows that are applied to the input image to detect the faces.
The detector will return the coordinates of the box that contains the face. When printing
the result, we see that it's a dictionary, where r represents the row position of the top left corner
of the detected window, c is the column position pixel, width is width of detected window, and
height, the height of the detected window. We'll use a function that shows the original image
with the detected face marked with red lines, forming a box containing the face.

84
 Code:
# Apply detector on the image
detected = detector.detect_multi_scale(img=image,
scale_factor= 1.2,
step_ratio= 1,
min_size=(10, 10),
max_size=( 200,
200))
print(detected)
# Show image with detected face marked
show_detected_face(image, detected)
Detected face: [{'r': 115, 'c': 210, 'width': 167, 'height': 167}]

Show detected faces: With this function I draw a rectangle around detected faces. We
won't discuss this function in detail here.
Code:
def show_detected_face(result, detected, title="Face image"):
plt.imshow(result)
img_desc = plt.gca()
plt.set_cmap('gray')
plt.title(title)
plt.axis('off')
for patch in detected:
img_desc.add_patch(
patches.Rectangle(
(patch['c'], patch['r']),
85
 patch['width'],
patch['height'],
fill= False,color='r',linewidth=2)
)
plt.show()

3.8. Real-world applications

1.7.35.Applications

Some cases where we might need to combine several techniques are, for example,
converting to images to grayscale before detecting edges or corners. Or, detecting faces to later
on blur them by applying a gaussian filter. And even reducing noise and restoring a damaged
image.

1.7.36.Privacy protection

Let's look at how we would solve a privacy protection case by detecting faces and then
anonymizing them. We'll use this image to work with.

86
 So for this case in particular, we'll first need to detect faces, using the cascade of
classifiers detector and then apply a gaussian filter to the cropped faces.
# Import Cascade of classifiers and gaussian filter
from skimage.feature import Cascade
from skimage.filters import gaussian
So, with the face detector initialized and ready to use, we can start detecting faces. For
each detected face, as the variable d, in the detected list, we'll use the coordinates to crop it out
of the image, in other words, extract it.
# Detect the faces
detected = detector.detect_multi_scale(img=image,
scale_factor= 1.2,
step_ratio=1,
min_size=( 50, 50),
max_size=(100, 100))
# For each detected face
for d in detected:
# Obtain the face cropped from detected coordinates
face = getFace(d)
This get Face function crops the face out of the image, using the detected face dictionary
that contains the coordinates. Then, we'll draw a rectangle around the detected face of the
image. Taking r, which is the row position of top left corner of detected rectangle as the X
starting position and c, which is the column as the starting Y position. These are the points
from where we'll now add a width and a height to complete the rectangle. And that's exactly
what we do next, we add the height and width to the starting points. So we have the rectangle
dimensions in the image. We then specify these dimensions in the original image from which
the face was detected so we can crop it, or slice it.
def getFace(d):
''' Extracts the face rectangle from the image using the
coordinates of the detected.'''
# X and Y starting points of the face rectangle
x, y = d['r'], d['c']
# The width and height of the face rectangle
87
 width, height = d['r'] + d['width'], d['c'] + d['height']
# Extract the detected face
face= image[x:width, y:height]
return face
Now that the face is cropped from the image, we'll apply the gaussian filter to blur it and
make it unrecognizable. This resulting image is assigned to the gaussian_face variable. As a
last step, we'll merge the blurry face back to the image, using another function called merge
blurry face.
# Detect the faces
detected = detector.detect_multi_scale(img=image,
scale_factor= 1.2,
step_ratio=1,
min_size=( 50, 50),
max_size=(100, 100))
# For each detected face
for d in detected:
# Obtain the face cropped from detected coordinates
face = getFace(d)
# Apply gaussian filter to extracted face
gaussian_face = gaussian(face, multichannel=True, sigma =
10)
# Merge this blurry face to our final image and show it
resulting_image = mergeBlurryFace(image, gaussian_face)

To do so, we again specify the starting X and Y points as well as the width and height, to
slice the original image. Then, the blurry face is assigned to the result
def mergeBlurryFace(original, gaussian_image):
# X and Y starting points of the face rectangle
x, y = d['r'], d['c']
# The width and height of the face rectangle
width, height = d['r'] + d['width'], d['c'] + d['height']
original[ x:width, y:height] = gaussian_image
88
 return original

So it results in an image that no longer contains people's faces in it and in this way,
personal data is anonymized. We need to remember that the classifier was only trained to
detect the front side of faces, not profile faces. So, if someone is turning the head too much to a
side, it won't recognize it. If you wnat to do that, you'll need to train the classifier with xml
files of profile faces, that you can find available online. Like some provided by the OpenCV
image processing library.

1.7.37.More cases

Just like the cases mentioned before, there are many more where you will we be able to
combine the processes that you learned during the course

89
EXERCISE:
Exercise 1: Finding the edges with Canny
Question: In this exercise you will identify the shapes in a grapefruit image by detecting
the edges, using the Canny algorithm.

Image preloaded as grapefruit.

The color module has already been preloaded for you

Code:

# Import the canny edge detector

from skimage.____ import ____
# Convert image to grayscale
grapefruit = ____(____)
# Apply canny edge detector
canny_edges = ____
# Show resulting image
show_image(canny_edges, "Edges with Canny")
Instruction:

 Import the canny edge detector from the feature module.

 Convert the image to grayscale, using the method from the color module used in previous
chapters.
 Apply the canny edge detector to the grapefruit image.

90
Exercise 2: Less edgy
Question: Let's now try to spot just the outer shape of the grapefruits, the circles. You can
do this by applying a more intense Gaussian filter to first make the image smoother. This can
be achieved by specifying a bigger sigma in the canny function.

In this exercise, you'll experiment with sigma values of the canny() function.

Image preloaded as grapefruit.

The show_image has already been preloaded

Code:

# Apply canny edge detector with a sigma of 1.8

canny_edges = canny(grapefruit, sigma=____)
Instruction

 Apply the canny edge detector to the grapefruit image with a sigma of 1.8.
 Apply the canny edge detector to the grapefruit image with a sigma of 2.2.
 Show the resulting images.
Exercise 3:
Question: In this exercise, you will detect the corners of a building using the Harris
corner detector.

91
 Image preloaded as building_image.

The functions show_image() and show_image_with_corners() have already been

preloaded for you. As well as the color module for converting images to grayscale

Code:

# Import the corner detector related functions and module

from skimage.____ import ____, corner_peaks
# Convert image from RGB-3 to grayscale
building_image_gray = ____
# Apply the detector to measure the possible corners
measure_image = ____
# Find the peaks of the corners using the Harris detector
coords = ____(____, min_distance=2)
# Show original and resulting image with corners detected
show_image(building_image, "Original")
show_image_with_corners(building_image, coords)
Instruction:

 Import the corner_harris() function from the feature module.

 Convert the building_image to grayscale.

 Apply the harris detector to obtain the measure response image with the possible corners.
 Find the peaks of the corners
92
Exercise 4: Less corners
Question: In this exercise, you will test what happens when you set the minimum distance
between corner peaks to be a higher number. Remember you do this with
the min_distance attribute parameter of the corner_peaks() function.

Image preloaded as building_image.

The functions show_image(), show_image_with_corners() and required packages have

already been preloaded for you. As well as all the previous code for finding the corners. The
Harris measure response image obtained with corner_harris() is preloaded as measure_image

Code:

# Find the peaks with a min distance of 2 pixels

coords_w_min_2 = ____(measure_image, min_distance=____)

93
 print("With a min_distance set to 2, we detect a total", len(coords_w_min
_2),

"corners in the image.")

Instruction

 Find the peaks of the corners with a minimum distance of 2 pixels.

 Find the peaks of the corners with a minimum distance of 40 pixels.

 Show original and resulting image with corners detected

Exercise 5: Face detection

Question: In this exercise, you will check whether or not there is a person present in an
image taken at night.

Image preloaded as night_image.

94
 The Cascade of classifiers class from feature module has been already imported. The
same is true for the show_detected_face() function, that is used to display the face marked in
the image and crop so it can be shown separately.

Code:

# Load the trained file from data

trained_file = ____.lbp_frontal_face_cascade_filename()
# Initialize the detector cascade
detector = ____(____)
# Detect faces with min and max size of searching window
detected = detector.detect_multi_scale(img = night_image,
scale_factor=1.2,
step_ratio=1,
min_size=(____),
max_size=(____))
# Show the detected faces
show_detected_face(night_image, detected)
Instruction:

 Load the trained file from the data module.

 Initialize the detector cascade with the trained file.

 Detect the faces in the image, setting the minimum size of the searching window to 10
pixels and 200 pixels for the maximum.

Exercise 6: Multiple faces

Question: In this exercise, you will detect multiple faces in an image and show them
individually. Think of this as a way to create a dataset of your own friends' faces!

95
 Image preloaded as friends_image.

The Cascade of classifiers class from feature module has already been imported, as well
as the show_detected_face() function which is used to display the face marked in the image
and crop it so it can be shown separately.

Code:

# Load the trained file from data

trained_file = ____.___()
# Initialize the detector cascade
detector = ____
# Detect faces with scale factor to 1.2 and step ratio to 1
detected = detector.____(img=friends_image,
scale_factor=____,
step_ratio=____,
min_size=(10, 10),
max_size=(200, 200))
# Show the detected faces
show_detected_face(friends_image, detected)
Instruction:

 Load the trained file .lbp_frontal_face_cascade_filename(). From the

data module

 Initialize the detector cascade with trained file

 Detect the faces in the image, setting a scale_factor of 1.2 and step_ratio of 1
96
Exercise 7: Segmentation and face detection
Question: Previously, you learned how to make processes more computationally efficient
with unsupervised superpixel segmentation. In this exercise, you'll do just that!

Using the slic() function for segmentation, pre-process the image before passing it to
the face detector.

Image preloaded as profile_image.

The Cascade class, the slic() function from segmentation module, and the
show_detected_face() function for visualization have already been imported. The detector is
already initialized and ready to use as detector.

Code:

# Obtain the segmentation with default 100 regions

segments = ____
# Obtain segmented image using label2rgb
segmented_image = ____(____, ____, kind='avg')
# Detect the faces with multi scale method
detected = detector.____(img=____,
97
 scale_factor=1.2,
step_ratio=1,
min_size=(10, 10), max_size=(1000, 1000))
# Show the detected faces
show_detected_face(segmented_image, detected)
Instruction:

 Apply superpixel segmentation and obtain the segments a.k.a. labels using slic()

 Obtain the segmented image using label2rgb(), passing the segments and
profile_image

 Detect the faces, using the detector with multi scale method.

Exercise 8: Privacy protection

Question: Let's look at a real-world application of what you have learned in the course.

In this exercise, you will detect human faces in the image and for the sake of privacy,
you will anonymize data by blurring people's faces in the image automatically.

Image preloaded as group_image.

You can use the gaussian filter for the blurriness.

The face detector is ready to use as detector and all packages needed have been imported

Code:

# Detect the faces

detected = ____.____(img=____,
scale_factor=1.2, step_ratio=1,
min_size=____, max_size=(100, 100))
98
 # For each detected face
for d in ____:
# Obtain the face rectangle from detected coordinates
face = getFaceRectangle(d)
# Apply gaussian filter to extracted face
blurred_face = ____(face, multichannel=____, sigma = ____)
# Merge this blurry face to our final image and show it
resulting_image = mergeBlurryFace(group_image, blurred_face)
show_image(resulting_image, "Blurred faces")
Instruction:

 Detect the faces in the image using the detector, set the minimum size of the searching
window to 10 by 10 pixels.
 Go through each detected face with a for loop.

 Apply a gaussian filter to detect and blur faces, using a sigma of 8.

Exercise 9: Help Sally restore her graduation photo

Question: You are going to combine all the knowledge you acquired throughout the
course to complete a final challenge: reconstructing a very damaged photo.

Help Sally restore her favorite portrait which was damaged by noise, distortion, and
missing information due to a breach in her laptop.

Sally's damaged portrait is already loaded as damaged_image.

You will be fixing the problems of this image by:

99
 Rotating it to be uprightusing rotate()

 Applying noise reduction with denoise_tv_chambolle()

 Reconstructing the damaged parts with inpaint_biharmonic() from the

inpaint module.

 show_image() is already preloaded.

Code:

# Import the necessary modules

from skimage.restoration import denoise_tv_chambolle, ____
from skimage import transform
# Transform the image so it's not rotated
upright_img = ____(damaged_image, 20)
# Remove noise from the image, using the chambolle method
upright_img_without_noise = ____(upright_img,weight=0.1, multichannel
=True)
# Reconstruct the image missing parts
mask = get_mask(upright_img)
result = ____.____(upright_img_without_noise, mask, multichannel=True)
show_image(result)
Instruction:

 Import the necessary module to apply restoration on the image.

 Rotate the image by calling the function rotate().

 Use the chambolle algorithm to remove the noise from the image.
 With the mask provided, use the biharmonic method to restore the missing parts of the
image and obtain the final image

100