Total Variation-Regularized Weighted Nuclear Norm Minimization For Hyperspectral Image Mixed Denoising
Total Variation-Regularized Weighted Nuclear Norm Minimization For Hyperspectral Image Mixed Denoising
Zhaojun Wu
Qiang Wang
Zhenghua Wu
Yi Shen
Abstract. Many nuclear norm minimization (NNM)-based methods have been proposed for hyperspectral image
(HSI) mixed denoising due to the low-rank (LR) characteristics of clean HSI. However, the NNM-based methods
regularize each eigenvalue equally, which is unsuitable for the denoising problem, where each eigenvalue
stands for special physical meaning and should be regularized differently. However, the NNM-based methods
only exploit the high spectral correlation, while ignoring the local structure of HSI and resulting in spatial dis-
tortions. To address these problems, a total variation (TV)-regularized weighted nuclear norm minimization
(TWNNM) method is proposed. To obtain the desired denoising performance, two issues are included. First,
to exploit the high spectral correlation, the HSI is restricted to be LR, and different eigenvalues are minimized
with different weights based on the WNNM. Second, to preserve the local structure of HSI, the TV regularization
is incorporated, and the alternating direction method of multipliers is used to solve the resulting optimization
problem. Both simulated and real data experiments demonstrate that the proposed TWNNM approach produces
superior denoising results for the mixed noise case in comparison with several state-of-the-art denoising meth-
ods. © 2016 SPIE and IS&T [DOI: 10.1117/1.JEI.25.1.013037]
Keywords: hyperspectral image denoising; low rank; total variation; nuclear norm minimization.
Paper 15611 received Jul. 29, 2015; accepted for publication Jan. 27, 2016; published online Feb. 26, 2016.
*Address all correspondence to: Zhaojun Wu, E-mail: [email protected] 1017-9909/2016/$25.00 © 2016 SPIE and IS&T
The UINTA can be used not only for the gray image but also be treated differently. (2) The NNM-based hyperspectral
for the sequence of images in a coupled manner, by treating denoising models can only take advantage of the high cor-
the combination of three images as a vector image and relation in spectral domain but ignore the spatial structure of
searching for the probability density functions in the prob- the HSI, thus resulting in some distortions and blurs in geo-
ability space. However, the UINTA may only use the local metrical structure.
correlation of the spectral bands in the HSI dataset. More- More recently, to narrow the big gap between the NNM
over, some volumetric data denoising methods, such as and the rank minimization, the weighted nuclear norm
BM4D,20 which is based on the similarity of cubes of minimization (WNNM) model has been proposed and
voxel (i.e., 3-D-patch-based) and Wiener filtering, can successfully applied in matrix completion32 and natural
also be used for the HSIs denoising. However, these patch- image denoising problems.33 However, few studies have
similarity-based methods tend to oversmooth the images and investigated the WNNM model for the HSI mixed denoising
the fine details are usually absent. problem. On the other hand, the total variation (TV) regu-
Recently, low-rank (LR)-based methods have attracted larization has been widely adopted in the hyperspectral
many researchers’ attentions. These methods are based on denoising problem,34,35 showing excellent performance in
the assumption that the clean hypersepctral image can be preserving the geometrical structure. However, no literature
presented by a linear combination of finite numbers of pure uses both the WNNM model and TV regularization for
endmembers, which indicates the LR characteristics in the hyperspectral denoising simultaneously.
corresponding Casorati matrix (a matrix whose columns Motivated by the excellent performance of the WNNM
consist of vectorized bands of the HSI). The low-rank model and the widely used TV regularization method, this
matrix recovery (LRMR) model was first introduced by paper proposes a hypersepctral image mixed denoising
Wright et al.21 as a “robust principal component analysis” model based on WNNM and TV regularization, termed
(RPCA) problem, aiming at recovering the LR matrix L the TV-regularized weighted nuclear norm minimization
from the corrupted observations D ¼ L þ S. Here, the cor- (TWNNM). Note that the proposed model is related to
rupted matrix S is unknown and assumed to be sparse. Zhang the existing work by Peng et al.36 but fundamentally different
et al.22 used the LRMR model for HSI mixed denoising and from that. In Peng’s work, only the WNNM model is con-
achieved a promising performance. The LR-based denoising sidered, ignoring the geometrical structure of HSI, and the
model with the data fidelity of F-norm can be denoted as model is only suitable for large sparse random noise but
not for the dense noise and mixed noise. Compared with
min rankðLÞ þ λkSk0 s:t: kD − L − Sk2F < δ; (1) the existing literature, the main contributions this paper can
L;S
EQ-TARGET;temp:intralink-;e001;63;455
Casorati matrix is denoted as DMx My ×Mz . As we only consider where λ1 and λ2 are two positive constants used to control
the additive and impulsive mixed noise, the denoising model the relative weights among the NNM, l1 -norm, and the
can be formulated as F-norm, and the parameter λ2 is related to the standard
deviation of Gaussian noise N. Then, the NNM problem
D ¼ L þ S þ N;
EQ-TARGET;temp:intralink-;e002;63;719 (2) can be easily solved by SVT under the ADMM28 framework.
Equation (6) is first converted to the following unconstrained
where LMx My ×Mz is the clean HSI, SMx My ×Mz denotes the function, written as
intense and sparse noise, such as the salt and pepper impulse
noise, and N Mx My ×Mz indicates the weak and dense noise,
L; ^ N^ ¼ arg min kLk þ λ1 kSk1 þ λ2 kNk2F
^ S;
such as Gaussian noise. Based on the assumption of linear L;S;N
EQ-TARGET;temp:intralink-;e007;326;675
2
mixture model, the size of clean HSI is far larger than its μ
rank, indicating the LR property of clean HSI. However, þ kD − L − S − N − Y 1 k2F ; (7)
2
there is no correlation among SMx My ×Mz and N Mx My ×Mz in
different bands due to stochastic property of noise. Thus, where the parameter μ is a positive constant used to control
the ranks of noisy components SMx My ×Mz and N Mx My ×Mz are the step size of the iteration, and Y 1 is the Lagrange multi-
usually full. The purpose of HSI mixed denoising is to plier. Then, the solution to Eq. (7) can be obtained by solving
restore the clean LR part LMx My ×Mz from the observed noisy the following two subproblems:
dataset DMx My ×Mz .
L^ ¼ arg minkLk þ μ2 kL − ðD − S − N − Y 1 Þk2F ; (8)
L
EQ-TARGET;temp:intralink-;e008;326;565
L;
^ S;
St ¼ arg min kD − Lt − Sk2F s:t: cardðSÞ ≤ k: (5) L;S;N 2
S
EQ-TARGET;temp:intralink-;e005;63;290
s:t: D ¼ L þ S þ N; (10)
These two subproblems can be solved by the LR approxi-
mation method and entrywise hard threshold method, respec- where the kLkw; stands for the weighted nuclear norm and
tively. Nevertheless, the LRMF-based method for the LRMR can be defined as
mixed denoising model still suffers from some weaknesses. X
There is no edge constraint imposed on the HSI, which may kLkw; ¼ kωi σ i ðLÞk1 :
EQ-TARGET;temp:intralink-;e011;326;220 (11)
cause some distortions in the spatial domain. In addition, the i
parameters of rank r and cardinality k should be predicted
accurately, which is very difficult for real application. In Eq. (11), σ i ðLÞ is the i’th eigenvalue of L and ωi is
The other approach for solving the LRMR problem is the a non-negative weight corresponding to σ i ðLÞ.
NNM method. To solve the problem [Eq. (3)], the NP-hard In general, the WNNM is not convex. However, regarding
problem is relaxed to the NNM problem by replacing the the special denosing model case, the WNNM can still be
rank minimization item with the nuclear norm, which can solved efficiently. It is known that the eigenvalues of the
be formulated as matrix L are in a nonascending order, and the large eigen-
values contribute to the main characteristics of the clean HSI
^ N^ ¼ arg min kLk þ λ1 kSk1 þ λ2 kNk2F ;
image while the small eigenvalues are always due to the
EQ-TARGET;temp:intralink-;e006;63;115
L;
^ S;
noisy part. Thus, we should shrink the larger eigenvalues
L;S;N 2
less, i.e., assigning smaller weights to the larger eigenvalues.
s:t: D ¼ L þ S þ N; (6) When all the weights are in a nondescending order, the
WNNM can come to a fixed point based on the following data fidelity between the noisy image and the original
Corollary, and the detailed proof of Corollary 1 could be clean image. λ is a positive constant, which controls the rel-
found in Gu’s work.33 ative weight between the data fidelity item kD − Lk2F and
regularization item RðLÞ. Generally, there are two kinds
Corollary 1 If the weights are non-negative and in a non-
of TV regularization, i.e., anisotropic TV and isotropic TV.
descending order, the WNNM problem can be solved by
Experimentally, the anisotropic TV will be superior for edges
an iterative process and the LR part L^ will have a fixed preserving to the isotropic TV and is employed in this paper.
point as L^ ¼ UDω ðΣÞV T . The Σ is the eigenvalue matrix By incorporating the TV regularization with the WNNM
of L obtained by SVD, and the Dω ðΣÞ means a weighted framework, we get the proposed TWNNM HSI denoising
shrinking operation to the eigenvalue matrix according to model, denoted as
the given weights ω, i.e., Dω ðΣÞ ¼ diag½ðσ i − ωi Þþ .
^ N^ ¼ arg min kLkw; þ λ1 kSk1 þ λ2 kNk2F þ λ3 RðLÞ;
EQ-TARGET;temp:intralink-;e014;326;642
fJ;
^ L;
^ S;
^ N;
^ Z^ h ; Z^ v g ¼ arg min kJkw; þ λ1 kSk1 We first apply the SVD to the ðL þ Y 2 Þ item, i.e.,
J;L;S;N;Zh ;Zv
EQ-TARGET;temp:intralink-;e020;63;730
λ2 ðU; Σ; VÞ ¼ SVDðL þ Y 2 Þ:
EQ-TARGET;temp:intralink-;e027;326;712 (27)
þ kNk2F þ λ3 ðkZh k1 þ kZv k1 Þ
2 Then, the weighted shrinking operator Dω ðΣÞ ¼
μ μ
þ 1 kD − L − S − N − Y 1 Þk2F þ 2 kJ − L − Y 2 k2F diag½ðσ i − ωi Þþ is applied to the eigenvalue matrix
2 2 Σ, where the σ i is the eigenvalue vector and ωi is
μ3 μ3
þ kZh − Dh ðLÞ − Y 3 kF þ kZv − Dv ðLÞ − Y 4 k2F ;
2 the corresponding weight; the detail can be referred
2 2 to Corollary 1 in Sec. 2.3. The estimated matrix J^ can
(20) be obtained by
where the Y 1 , Y 2 , Y 3 , and Y 4 are four Lagrange multipliers. J^ ¼ UDωμ−1 ½ΣV T : (28)
To solve the unconstrained problem [Eq. (20)], we carry out
EQ-TARGET;temp:intralink-;e028;326;614
2
N,
^ which is denoted as
(21)
N^ ¼ λ2μþμ
1
ðD − L − S − Y 1 Þ: (30)
μ
EQ-TARGET;temp:intralink-;e030;326;414
1
Z^ v ¼ arg minλ3 kZv k1 þ 3 kZv − ðDv ðLÞ þ Y 4 Þk2F .
Zv
EQ-TARGET;temp:intralink-;e022;63;396
2
This step is specified in lines 13 to 14 in
(22) Algorithm 1.
Step 6: The problem of L^ item is also a quadratic optimi-
Then, Eqs. (21) and (22) can be solved by zation problem, which is formulated as
applying the soft-thresholding (shrinkage) operator
denoted as
L^ ¼ arg min fðLÞ
EQ-TARGET;temp:intralink-;e031;326;335
L
Z^ h ¼ S λ3 μ−1 ½Dh ðLÞ þ Y 3 ; (23) μ1
EQ-TARGET;temp:intralink-;e023;63;315
3
½Dv ðLÞ þ Y 4 ; (24) 2 2
μ3
þ kZv − Dv ðLÞ − Y 4 k2F : (31)
where the shrinkage operator is S ε ½x ¼ sgnðxÞ: 2
maxðjxj − ε; 0Þ. The definitions of Dh and Dv can
be found in Eqs. (16) and (17). This step is specified Set the partial derivative of fðLÞ as zero, i.e.,
in lines 5 to 7 in Algorithm 1.
∂fðLÞ
EQ-TARGET;temp:intralink-;e032;326;219
μ
S^ ¼ arg minλ1 kSk1 þ 1 kS − ðD − L − N − Y 1 Þk2F þ μ3 DT v ðDv ðLÞ − Zv þ Y 4 Þ
S 2
¼ S λ1 μ−1
1
½D − L − N − Y 1 : (25) ¼ 0; (32)
This step is specified in lines 8 to 9 in Algorithm 1. where the DT h and DT v are two conjugate operators
Step 4. In this step, we get the solution J^ item by applying of TV in horizontal and vertical directions. For an
the WNNM model, and the corresponding problem image I ∈ RðMx ×My Þ , the DT h and DT v operators can
can be denoted as be defined as Eqs. (33) and (34), respectively.
Algorithm 1 HSI Denoising via TWNNM and ADMM. Iði; j − 1Þ − Iði; jÞ 1
DT h ½Iði; jÞ ¼ ; (33)
0 j¼1
EQ-TARGET;temp:intralink-;e033;326;752
Input: D; λ1 ; λ2 ; λ3
Step 1: Initialization: μ1 ¼ μ2 ¼ μ3 ¼ 0.5; Iði − 1; jÞ − Iði; jÞ 1
1: DT v ½Iði; jÞ ¼ : (34)
γ 1 ¼ γ 2 ¼ γ 3 ¼ γ 4 ¼ 1, 0 i¼1
EQ-TARGET;temp:intralink-;e034;326;714
ð0Þ ð0Þ
2: Lð0Þ ¼ S ð0Þ ¼ N ð0Þ ¼ Z h ¼ Z v ¼ J ð0Þ ¼ 0; When using the TV and its conjugate operations,
ð0Þ ð0Þ ð0Þ ð0Þ
Y1 ¼ Y2 ¼ Y3 ¼ Y4 ¼ 0 the hyperspectral Casorati matrix will be rearranged
into 3-D cube data again, and then the TV and its
3: ω0i ¼ 1, ε ¼ 10−6 , max iter ¼ 50, tol ¼ 10−6 , k ¼ 0 conjugate operators are applied to each band image
in the spatial domain. The resulting dataset will then
4: for k ¼ 0;1; : : : ; max iter be arranged to its Casorati matrix form for the sub-
sequent iteration. In order to solve Eq. (32), the pre-
5: Step 2: Fix the others and update Z h and Z v by conditioned conjugate gradients (PCG) method39 is
adopted. In the implementation, we use the “PCG”
6: Z h ¼ Sλ3 μ−1 ½D h ðLÞ þ Y 3 function in MATLAB R2014b. Please refer to the
3
ðkþ1Þ
10: Step 4: Fix the others and update J by EQ-TARGET;temp:intralink-;e035;326;510
ωi ¼ σ ðLðkÞ
1
Þþε
: (35)
i
used to control the relative contribution of l1 -norm, and a 4.1 Simulated Data Experiments
larger value will lead to sparser matrix SMx My ×Mz . In all 4.1.1 Data and experimental settings
the experiments, the reflectance values of all the HSI are lin-
We adopt two HSI datasets, the Washington DC Mall (WDC
early scaled to the range of [0,1]. In this case, we set λ1 ¼
for short) dataset,41 and the Pavia city center (PAVIA for
range∕10 ¼ 0.1 empirically and we find it works well for all
short) dataset42 in the simulated experiments. The whole
the experiments.
image of WDC contains 1208 × 307 pixels ranging from
As to the proposed TWNNM model in Eq. (18), we have
400-nm to 2500-nm spectral bands, and only a subimage
the constraint condition that D ¼ L þ S þ N, so the λ22 kNk2F with the size of 256 × 256 × 191 is used in the experiment.
item can also be denoted as The PAVIA dataset was collected by the ROSIS-03, and the
first 22 bands of this data are removed due to the heavy
λ2
kNk2F ¼ λ22 kD − L − Sk2F : (36) noise; thus, the selected subimage in our experiment is of
2
size 216 × 216 × 80. In order to evaluate the experimental
EQ-TARGET;temp:intralink-;e036;63;650
noise variance σ 2 in our experiments ranges from 0.07 to HSI patches and the corresponding eigenvalues are obtained,
0.13, and we adopt the average value 0.1). Meanwhile, in as shown in Fig. 2. In order to take a closer look at the figure,
the TWNNM method, the weight vector ωi is set based on we magnify the region marked with pink dashed in two mag-
Eq. (12), and the TV-regularization parameter λ3 is set to nifications. It should be noted that in Fig. 2, the NNM-based
0.002 and 0.004 for WDC and PAVIA, respectively. method shrinks the singular values more heavily, resulting in
Please refer to Sec. 3.4 for more details about the parameter overshrink. However, in the WNNM-based method, the sin-
determination. In both the WDC and PAVIA cases, the gular values of denoised HSI are more approximate to the
experiments are conducted for 50 times and the average clean one. Moreover, from Table 1, we also find that the pro-
results are adopted. posed TWNNM method can get a superior result to other
methods in the experiment.
4.1.2 Experimental Results on Washington, DC, Then, the PSNR and SSIM values of each band with dif-
Mall Case ferent denoising methods are calculated and shown in Fig. 3.
The first simulated dataset is WDC, and the MPSNR, As can be seen, the PSNR and SSIM values obtained by the
MSSIM, and MSAD of the denoising results are shown in proposed TWNNM method are higher than those of the other
Table 1. It is clear that the WNNM-based method shows methods in almost all bands, especially in bands 25 to 75.
a great superiority to the NNM-based method, which mainly This result can also be explained theoretically, as in the
benefits from the more suitable shrinking strategy of TWNNM method, the TV-regularization parameter λ3 is
different eigenvalues. As for the WNNM-based method, set to the same value in all bands. However, this may not
the larger eigenvalue should be shrunk less and the smaller be the most appropriate strategy for HSI denoising, as the
eigenvalue should be shrunk more. To illustrate the effect of noise intensities in different bands are different. To overcome
different shrinking strategies between the NNM-based and this weakness, we can use the adapted TV model as the regu-
WNNM-based methods, we take a patch of image with larization item instead of the fixed parameter TV regulariza-
the size of 20 × 20 × 191 from the clean HSI, the noisy tion. However, to simplify the computation, we still adopt
HSI, the NNM denoised HSI, and the WNNM denoised the fixed parameter TV regularization in this paper, and the
HSI, respectively, and then reshape them into a correspond- adapted parameter TV regularization will be studied in future
ing Casorati matrix. Then, the SVD is conducted on the four research.
70
CLEAN A synthetic image with only Gaussian noise and its
64 NOISE
60 NNM denoised images are shown in Fig. 4, and Fig. 5 presents
WNNM
62 magnified parts of the results in Fig. 4 marked with the
50
60 red rectangle. In Fig. 4, it is clear that the GSM cannot effec-
Singular Values
40 0 1 2
tively suppress the Gaussian noise and also cause some
62.1 artifacts, and the BM4D tends to oversmooth the image and
30
62.05 most of the details are lost. As for Figs. 4(e)–4(h), it is hard to
20 62 distinguish their differences due to the low spatial resolution.
0.98 1 1.02
However, from the magnified results in Fig. 5, it can be
10
seen that the LRMR and NNM methods fail to remove some
0
large Gaussian noise and fine details are lost. Compared
0 20 40 60 80 100 120 140 160 180 200
Index with LRMR and NNM, the WNNM and TWNNM remove
most of the noise and the fine details are retained. We can
Fig. 2 The singular values of the HSI patches. also observe that the edges of image in Fig. 5(h) are more
40 1
0.9
35
0.8
30 0.7
PSNR
SSIM
0.6 0.97 GSM
25 37.5 GSM BM4D
BM4D 0.5 0.96 LRMR
37 LRMR NNM
20 NNM 0.4 WNNM
WNNM 0.95 TWNNM
36.5
TWNNM 0.3 146 148 150 152
15 140 142 144 146 148
0.2
0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200
Band number Band number
(a) (b)
Fig. 3 PSNR and SSIM values of each band in WDC case: (a) PSNR value and (b) SSIM value.
Fig. 4 Denoising results of WDC image in Gaussian noise case: (a) original band 45; (b) noisy band;
(c) GSM; (d) BM4D; (e) LRMR; (f) NNM; (g) WNNM; and (h) TWNNM.
Fig. 5 Magnified results of Fig. 4: (a) original band 45; (b) noisy band; (c) GSM; (d) BM4D; (e) LRMR;
(f) NNM; (g) WNNM; and (h) TWNNM.
clear than that in Fig. 5(g), which mainly benefits from the rectangle, respectively. As can be seen, the GSM method
TV regularization for edges preserving in our proposed performs badly and blurs the image. Meanwhile, the BM4D
TWNNM. blurs the edges and the fine details are lost. As to the
Figure 6 displays the denoising results of the mixture of magnified image in Fig. 7, it is shown that our proposed
Gaussian and salt and pepper impulse noise in band 26, and TWNNM has the best performance and can remove nearly
Fig. 7 shows the magnified parts of Fig. 6 marked with a red all the mixed noise, and the edges in Fig. 7(h) are preserved.
Fig. 6 Denoising results of WDC image in mixed noise case: (a) original band 26; (b) noisy band;
(c) GSM; (d) BM4D; (e) LRMR; (f) NNM; (g) WNNM; and (h) TWNNM.
Fig. 7 Magnified results of Fig. 6: (a) original band 26; (b) noisy band; (c) GSM; (d) BM4D; (e) LRMR;
(f) NNM; (g) WNNM; and (h) TWNNM.
The LRMR and NNM methods can also remove the mixed the noise-free and the denoising results45 of pixel (200,62).
noise, but there is still some dense noise in Figs. 7(e) and From Figs. 8 and 9 we can observe that the GSM method
7(f). As can be seen, the WNNM can remove the mixed fails to suppress the noise in the spectral domain; meanwhile,
noise more effectively than LRMR and NNM. However, the the BM4D causes heavy distortions, especially in bands
WNNM cannot effectively preserve the edges, as shown in 25 to 60. As for the LRMR and NNM methods, some noise
Fig. 7(g). still exists in the spectral domain, which can be observed in
To further demonstrate the performance of the proposed Figs. 8(e) and 8(f) within the marked rectangle. In Figs. 8(g)
TWNNM, we show the spectral signatures before and and 8(h), the denoising results in the spectral domain by
after denoising by comparing with other methods. Figure 8 the WNNM and TWNNM methods are very similar. The
shows the spectral signatures of pixel (200,62), and Fig. 9 main reason is that the denoising effect in spectral domain
presents the differences in the spectral signatures between is mostly due to the LR constraint of the images, and the
0 0 0 0
0 50 100 150 0 50 100 150 0 50 100 150 0 50 100 150
Band number Band number Band number Band number
(a) (b) (c) (d)
10000 10000 10000 10000
0 0 0 0
0 50 100 150 0 50 100 150 0 50 100 150 0 50 100 150
Band number Band number Band number Band number
(e) (f) (g) (h)
Fig. 8 Spectrum of pixel(200,62) in the denoised results: (a) original; (b) noisy; (c) GSM; (d) BM4D;
(e) LRMR; (f) NNM; (g) WNNM; and (h) TWNNM.
2000 2000
0 0
magnified regions. In Fig. 10, we can see that the proposed
–2000 –2000
TWNNM outperforms all the other methods in our compari-
0 50 100 150 0 50 100 150 son experiments in almost all bands. The GSM method per-
(a) (b) forms badly, especially in the mixed noise case. The LRMR
2000 2000
0 0
method performs well in Gaussian noise, but its performance
–2000 –2000
declines in the case of mixed noise, especially when the
0 50 100 150 0 50 100 150 image is polluted by serious mixed noise, as shown in
(c) (d) bands 22 to 29 in Fig. 10. Figures 11 and 13 show band
2000 2000
0 0
45 with only Gaussian noise and band 17 with mixed
–2000 –2000
noise, respectively. Figures 12 and 14 display the magnified
0 50 100 150 0 50 100 150 regions of the results in Figs. 11 and 13 marked with a red
(e) (f) rectangle, respectively. As shown in Figs. 11–14, we can
2000
see that the GSM method cannot effectively suppress the
0
noise and tends to oversmooth the image, especially under
–2000
0 50 100 150 the mixed noise case. Meanwhile, the BM4D performs
(g) well under Gaussian noise but fails to remove the mixed
noise, and the edges are oversmoothed. The LRMR also
Fig. 9 Difference between the clean spectrum and the denoised
results of pixel(200,62): (a) noisy; (b) GSM; (c) BM4D; (d) LRMR;
fails to restore the mixed noisy image, which can be seen
(e) NNM; (f) WNNM; and (g) TWNNM. Note that the horizontal axis in Figs. 13(e) and 14(e), as there are still many blurry
indicates the index of band number and the vertical axis indicates white dots in the two images. As for the NNM and
the value of digital number (DN). WNNM, the visual effect is quite similar in the selected
two bands in Figs. 11–14, even though the PSNR and
SSIM of the result obtained by the WNNM are higher than
WNNM and TWNNM methods hold the same LR constraint
that of NNM. However, compared with all other methods,
in spectral domain, thus resulting in nearly the same denois-
the proposed TWNNM shows superior performance against
ing result in the spectral domain.
mixed noise and simultaneously preserves the edges and
local details of the original image, which can be seen in
4.1.3 Experimental Results on PAVIA Case Figs. 11–14 clearly.
In the PAVIA case, we compare our proposed TWNNM with
the GSM, BM4D, LRMR, NNM, and WNNM. We calculate 4.1.4 Sensitivity Analysis of Parameter λ3
the MPSNR, MSSIM, and MSAD of different results, as To further explain the effect of parameter λ3 on the final
shown in Table 2. Then, the PSNR and SSIM of each denoising performance, we conduct two simulated experi-
band are also calculated, displayed in Fig. 10 with two ments on the WDC and PAVIA datasets. We first set the
38 1
36
0.9
34
32 0.8
30
PSNR
SSIM
0.7
28
37 0.98 GSM
GSM
26 0.6 BM4D
BM4D 0.97 LRMR
LRMR
24 36.5 NNM
NNM
0.96 WNNM
WNNM 0.5 TWNNM
22 TWNNM
36 57 58 59 60
60 61 62 63 64
20 0.4
0 10 20 30 40 50 60 70 80 0 10 20 30 40 50 60 70 80
Band number Band number
(a) (b)
Fig. 10 PSNR and SSIM values of each band in PAVIA case: (a) PSNR value and (b) SSIM value.
Fig. 11 Denoising results of PAVIA image in Gaussian noise case: (a) original band 45; (b) noisy band;
(c) GSM; (d) BM4D; (e) LRMR; (f) NNM; (g) WNNM; and (h) TWNNM.
λ3 to 0, and then carefully increase the value of λ3 . The final oversmoothing. In other words, the proposed method is sen-
denoised results are calculated in the MPSNR index, as sitive to λ3 , and this parameter is suggested to be smaller
shown in Fig. 15. The results indicate that too small a than 0.01.
value of parameter λ3 (e.g., 0.0005 in both WDC and
PAVIA cases) is inadequate to effectively suppress the 4.2 Real Data Experiments
noise or preserve the edges, while too large a value of λ3 In the real data experiments, the proposed TWNNM is tested
(e.g., 0.01 in both WDC and PAVIA cases) may lead to on two publicly available HSIs, Beltsville46 and Indian
Fig. 12 Magnified results of Fig. 11: (a) original band 45; (b) noisy band; (c) GSM; (d) BM4D; (e) LRMR;
(f) NNM; (g) WNNM; and (h) TWNNM.
Fig. 13 Denoising results of PAVIA image in mixed noise case: (a) original band 17; (b) noisy band;
(c) GSM; (d) BM4D; (e) LRMR; (f) NNM; (g) WNNM; and (h) TWNNM.
Pines,47 by comparing with the BM4D, LRMR, NNM, and respectively. In the NNM, WNNM, and TWNNM methods,
WNNM. Beltsville is the first dataset used in our real data the parameters are set to the same as that in simulated
experiments. It was acquired by the push-broom sensors experiment for WDC case. To simplify the computation,
and corrupted by both dead lines and random noise. For we only use a small part of Beltsville with the size of
the BM4D method, the noise variation is set to 13. As for 300 × 300 × 360. Due to the lack of a ground truth, we just
the LRMR, the rank and cardinality are set to 4 and 200, display the bands 188 and 345 of the HSI for visual quality
Fig. 14 Magnified results of Fig. 13: (a) original band 17; (b) noisy band; (c) GSM; (d) BM4D; (e) LRMR;
(f) NNM; (g) WNNM; and (h) TWNNM.
36 36.5
35.8
35.6 36
35.4
35.2 35.5
MPSNR
MPSNR
35
34.8 35
34.6
34.4 34.5
34.2
34 34
0.000 0.002 0.004 0.006 0.008 0.010 0.012 0.000 0.002 0.004 0.006 0.008 0.010 0.012 0.014
Parameter λ 3 Parameter λ 3
(a) (b)
Fig. 15 Sensitivity analysis of parameter λ3 in (a) WDC case and (b) PAVIA case.
comparison in Figs. 16 and 18, respectively. Figures 17 and pixels and 220 bands. The dataset contains much random
19 present the magnified parts of the results in Figs. 16 noise and about 20 noisy bands of water absorption; it
and 18 marked with the red rectangle. In Figs. 16–19(a), thus can be used for validating the techniques of HSI
both random noise and dead line noise are obvious. The denoising. For the BM4D method, the noise variation is
BM4D removes most of the noise but blurs the edges, set to 15. As for the LRMR, the rank and cardinality are
thus resulting in oversmoothing. As for the results of set to 4 and 200, respectively. For the NNM, WNNM,
LRMR and NNM in (c) and (d), the images are dominated and TWNNM methods, the parameters are set to the
by random noise. On the contrary, the results of WNNM in same as those in the simulated experiment for the WDC
(e) show a more competitive effect against the random case. The visual results of bands 15 and 220 are given in
noise. Comparing with all other methods, the proposed Figs. 20 and 22, respectively. The magnified parts of the
TWNNM shows superior performance against random noise results in Figs. 20 and 22 marked with a red rectangle
and dead line noise, and simultaneously preserves the edges are displayed in Figs. 21 and 23. In Figs. 20–23, we can
and textural details, which can be obviously observed in clearly see that the original images contain much random
Figs. 16–19(f). noise and the images seem too blurry for display. The
The second real dataset, Indian Pines, was acquired in BM4D method tends to blur the images and also causes
northwestern Indiana in 1992, with the size of 145 × 145 many artificial distortions, which can be found in
Fig. 16 Denoising results for band 188 of Beltsville: (a) original; (b) BM4D; (c) LRMR; (d) NNM;
(e) WNNM; and (f) TWNNM.
Fig. 17 Magnified results of Fig. 16: (a) original; (b) BM4D; (c) LRMR; (d) NNM; (e) WNNM; and
(f) TWNNM.
Fig. 18 Denoising results for band 345 of Beltsville: (a) original; (b) BM4D; (c) LRMR; (d) NNM;
(e) WNNM; and (f) TWNNM.
Figs. 20–23(b). The LRMR and NNM methods can sup- denoised images look more clear. Moreover, by comparing
press part of the noise, but there is still much noise Figs. 20–23(e) and 23(f), we can see that the edges and
in Figs. 20–23(c) and 23(d). On the contrary, the WNNM details in (f) are more clear than that of (e), demonstrat-
and TWNNM can suppress most of the noise, and the ing that the proposed TWNNM is much superior for
Fig. 19 Magnified results of Fig. 18: (a) original; (b) BM4D; (c) LRMR; (d) NNM; (e) WNNM; and
(f) TWNNM.
Fig. 20 Denoising results for band 15 of Indian Pines: (a) original; (b) BM4D; (c) LRMR; (d) NNM;
(e) WNNM; and (f) TWNNM.
hyperspectral denoising to WNNM and other methods in are given. The SVM with cross validation is adopted, and only
our comparison experiments. 11 classes of ground targets are considered in our experiment.
To further verify the performance of the proposed About 5% of the samples are used as training data, and the
TWNNM, the classification results of the Indian Pines dataset others used as testing data. Figure 24 shows the classification
Fig. 21 Magnified results of Fig. 20: (a) original; (b) BM4D; (c) LRMR; (d) NNM; (e) WNNM; and
(f) TWNNM.
Fig. 22 Denoising results for band 220 of Indian Pines: (a) original; (b) BM4D; (c) LRMR; (d) NNM;
(e) WNNM; and (f) TWNNM.
maps obtained before and after denoising with different NNM, and LRMR, and performs much better than the original
methods associated with the corresponding overall accuracy image and the BM4D method. The classification results also
(OA) scores in percent. As presented in Fig. 24, the proposed serve as an example, demonstrating the superiority of our
TWNNM tends to produce a slightly higher OA than WNNM, proposed TWNNM for real HSI denoising.
Fig. 23 Magnified results of Fig. 22: (a) original (band 220); (b) BM4D; (c) LRMR; (d) NNM; (e) WNNM;
and (f) TWNNM.
Fig. 24 Classification results of the Indian Pines dataset before and after denoising: (a) original;
(b) BM4D; (c) LRMR; (d) NNM; (e) WNNM; and (f) TWNNM.
model lies in the simultaneous usage of both the high spec- 12. P. Milanfar, “A tour of modern image filtering: new insights and meth-
ods, both practical and theoretical,” IEEE Signal Process. Mag. 30(1),
tral correlation and spatial structure constraint of HSI. To 106–128 (2013).
obtain the desired denoising performance, two issues are 13. J. Portilla et al., “Image denoising using scale mixtures of Gaussians in
considered. (1) The Casorati matrix of HSI is restricted the wavelet domain,” IEEE Trans. Image Process. 12(11), 1338–1351
(2003).
to be LR, based on the high spectral correlation of the 14. J. Guerrero-Colón et al., “Image restoration using space-variant
clean HSI. Then, the WNNM-based model is adopted to Gaussian scale mixtures in overcomplete pyramids,” IEEE Trans.
Image Process. 17(1), 27–41 (2008).
approximate the rank of clean HSI, which treats each eigen- 15. D. Letexier and S. Bourennane, “Multidimensional Wiener filtering
value of Casorati matrix differently and shrinks it with dif- using fourth order statistics of hyperspectral images,” in IEEE Int.
ferent weights. (2) The TV regularization is incorporated in Conf. on Acoustics, Speech and Signal Processing, pp. 917–920,
IEEE, Las Vegas, Nevada (2008).
the derived model, which is used to preserve the spatial 16. G. Chen, T. D. Bui, and A. Krzyzak, “Denoising of three-dimensional
structure of HSI, and the ADMM method is adopted to data cube using bivariate wavelet shrinking,” Int. J. Pattern Recognit.
solve the resulting optimization problem. The proposed Artif. Intell. 25(03), 403–413 (2011).
17. A. Brook, “Three-dimensional wavelets-based denoising of hyperspec-
TWNNM model is tested on two simulated datasets and tral imagery,” J. Electron. Imaging 24(1), 013034 (2015).
two real datasets, and the experimental results demonstrate 18. B. Rasti et al., “Hyperspectral image denoising using first order spectral
roughness penalty in wavelet domain,” IEEE J. Sel. Top. Appl. Earth
that our proposed TWNNM can suppress the mixture of Obs. Remote Sens. 7(6), 2458–2467 (2014).
Gaussian noise, salt and pepper impulse noise, and dead 19. S. P. Awate and R. T. Whitaker, “Unsupervised, information-theoretic,
line noise, while preserving the spatial structure and adaptive image filtering for image restoration,” IEEE Trans. Pattern
Anal. Mach. Intell. 28(3), 364–376 (2006).
local details of the HSI simultaneously. In addition, the pro- 20. M. Maggioni et al., “Nonlocal transform-domain filter for volumetric
posed TWNNM provides a mixed denoising framework, in data denoising and reconstruction,” IEEE Trans. Image Process. 22(1),
which the spectral correlation constraints, edge constraints, 119–133 (2013).
21. J. Wright et al., “Robust principal component analysis: exact recovery
and other prior knowledge (if available) can be conven- of corrupted low-rank matrices via convex optimization,” in Advances
iently incorporated. in Neural Information Processing Systems, pp. 2080–2088, Curran
Nevertheless, there still remain two possible extensions of Associates, Inc., Vancouver, B.C. (2009).
22. H. Zhang et al., “Hyperspectral image restoration using low-rank matrix
our proposed TWNNM model. (1) The parameters of the recovery,” IEEE Trans. Geosci. Remote Sens. 52(8), 4729–4743 (2014).
TV-regularization item in TWNNM is the same for all 23. A. M. Buchanan and A. W. Fitzgibbon, “Damped Newton algorithms
for matrix factorization with missing data,” in IEEE Computer Society
bands. However, as for the HSI, the noise levels are different Conf. on Computer Vision and Pattern Recognition, Vol. 2, pp. 316–
over bands and a noise-adjusted parameter setting strategy 322, IEEE, San Diego, California (2005).
should be studied in the future. (2) Although the TV regu- 24. V. C. Klema and A. J. Laub, “The singular value decomposition: its
computation and some applications,” IEEE Trans. Autom. Control
larization is effective for edge preserving, it may suffer from 25(2), 164–176 (1980).
the undesired staircasing artifacts. Thus, a more robust and 25. E. J. Candès et al., “Robust principal component analysis?,” J. ACM
effective regularization technique should be incorporated in 58(3), 11 (2011).
26. D. L. Donoho, M. Gavish, and A. Montanari, “The phase transition of
the model to overcome these drawbacks, which is the empha- matrix recovery from Gaussian measurements matches the minimax mse
sis of future work. of matrix denoising,” Proc. Natl. Acad. Sci. 110(21), 8405–8410 (2013).
27. J.-F. Cai, E. J. Candès, and Z. Shen, “A singular value thresholding algo-
rithm for matrix completion,” SIAM J. Optim. 20(4), 1956–1982 (2010).
Acknowledgments 28. S. Boyd et al., “Distributed optimization and statistical learning via
the alternating direction method of multipliers,” Found. Trends Mach.
Thanks to National Natural Science Foundation of China Learn. 3(1), 1–122 (2011).
(Grant No. 61174016) and (Grant No. 61171197) for funding. 29. W. Li et al., “Efficient completion for corrupted low-rank images via alter-
nating direction method,” J. Electron. Imaging 23(3), 033018 (2014).
30. H. Song, G. Wang, and K. Zhang, “Hyperspectral image denoising via
References low-rank matrix recovery,” Remote Sens. Lett. 5(10), 872–881 (2014).
31. Y. Zhao and J. Yang, “Hyperspectral image denoising via sparsity and
1. A. Zare et al., “Vegetation mapping for landmine detection using long- low rank,” in IEEE Int. Geoscience and Remote Sensing Symp.,
wave hyperspectral imagery,” IEEE Trans. Geosci. Remote Sens. 46(1), pp. 1091–1094, IEEE, Melbourne, Victoria (2013).
172–178 (2008). 32. J. Xu, “Reweighted nuclear norm minimization for matrix completion,”
2. J. M. Bioucas-Dias et al., “Hyperspectral remote sensing data analysis https://webspace.utexas.edu/jx598/www/Reweighted.pdf (2011).
and future challenges,” IEEE Geosci. Remote Sens. Mag. 1(2), 6–36 33. S. Gu et al., “Weighted nuclear norm minimization with application to
(2013). image denoising,” in IEEE Conf. on Computer Vision and Pattern
3. S. Serranti, R. Palmieri, and G. Bonifazi, “Hyperspectral imaging Recognition, pp. 2862–2869, IEEE, Columbus, Ohio (2014).
applied to demolition waste recycling: innovative approach for product 34. H. K. Aggarwal and A. Majumdar, “Exploiting spatiospectral correla-
quality control,” J. Electron. Imaging 24(4), 043003 (2015). tion for impulse denoising in hyperspectral images,” J. Electron.
4. J. P. Kerekes and J. E. Baum, “Hyperspectral imaging system model- Imaging 24(1), 013027 (2015).
ing,” Lincoln Lab. J. 14(1), 117–130 (2003). 35. Q. Yuan, L. Zhang, and H. Shen, “Hyperspectral image denoising
5. A. N. Zemliachenko et al., “Lossy compression of hyperspectral images employing a spectral–spatial adaptive total variation model,” IEEE
based on noise parameters estimation and variance stabilizing trans- Trans. Geosci. Remote Sens. 50(10), 3660–3677 (2012).
form,” J. Appl. Remote Sens. 8(1), 083571 (2014). 36. Y. Peng et al., “Reweighted low-rank matrix recovery and its application
6. A. Hagag, M. Amin, and F. E. A. El-Samie, “Simultaneous denoising in image restoration,” IEEE Trans. Cybern. 44(12), 2418–2430 (2014).
and compression of multispectral images,” J. Appl. Remote Sens. 7(1), 37. T. Zhou and D. Tao, “Godec: randomized low-rank & sparse matrix
073511 (2013). decomposition in noisy case,” Proc. of the 28th Int. Conf. on
7. S. Bourennane and C. Fossati, “Adaptive multidimensional Wiener Machine Learning, pp. 33–40 Omnipress (2011).
filtering for target detector improvement,” J. Appl. Remote Sens. 38. L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based
4(1), 043524 (2010). noise removal algorithms,” Phys. D 60(1), 259–268 (1992).
8. J. Chanussot, C. Collet, and K. Chehdi, Multivariate Image Processing, 39. R. Barrett et al., Templates for the Solution of Linear Systems: Building
Wiley-ISTE, London, UK (2009). Blocks for Iterative Methods, Vol. 43, Siam, Philadelphia, Pennsylvania
9. S. Paris et al., “A gentle introduction to bilateral filtering and its appli- (1994).
cations,” in ACM SIGGRAPH, ACM , New York (2007). 40. W. Guo, J. Qin, and W. Yin, “A new detail-preserving regularity
10. A. Buades, B. Coll, and J.-M. Morel, “A non-local algorithm for image scheme,” UCLA CAM Report, 13–04 (2013).
denoising,” in IEEE Computer Society Conf. on Computer Vision and 41. https://engineering.purdue.edu/biehl/MultiSpec/hyperspectral.html
Pattern Recognition, San Diego, California Vol. 2, pp. 60–65, IEEE 42. Paolo Gamba, “Pavia Centre scene,” (2014) http://www.ehu.eus/
(2005). ccwintco/index.php?title=Hyperspectral_Remote_Sensing_Scenes
11. K. Dabov et al., “Image denoising by sparse 3-D transform-domain 43. Z. Wang et al., “Image quality assessment: from error visibility to
collaborative filtering,” IEEE Trans. Image Process. 16(8), 2080–2095 structural similarity,” IEEE Trans. Image Process. 13(4), 600–612
(2007). (2004).
44. W. He et al., “Hyperspectral image denoising via noise-adjusted itera- been a professor with the Department of Control Science and
tive low-rank matrix approximation,” IEEE J. Sel. Top. Appl. Earth Engineering, HIT, since 2008. His current research interests include
Obs. Remote Sens. 8(6), 1–12 (2015). wireless sensor networks, signal and image processing, remote sens-
45. G. Chen and S.-E. Qian, “Denoising of hyperspectral imagery using ing, and intelligent detection technology.
principal component analysis and wavelet shrinkage,” IEEE Trans.
Geosci. Remote Sens. 49(3), 973–980 (2011).
46. SpecTIR, “Agriculture & vegetation sample: Beltsville, MD, USA,” Zhenghua Wu received his PhD degree in the Department of Control
(2012) http://www.spectir.com/free-data-samples/ Science and Engineering from Harbin Institute of Technology (HIT) in
47. M. F. Baumgardner, L. L. Biehl, and D. A. Landgrebe “220 Band 2015. He has been an intermediate principal engineer with the China
AVIRIS hyperspectral image data set: June 12, 1992 Indian Pine Electronics Technology Group Corporation No. 38 Research Institute,
Test Site 3,” (2013)https://purr.purdue.edu/publications/1947/1 Hefei, China, since 2015. His current research interests include
remote sensing and signal and image processing.
Zhaojun Wu is currently working toward his PhD degree in the
Department of Control Science and Engineering, Harbin Institute of Yi Shen received his BS, MS, and PhD degrees in the Department of
Technology, Harbin, China. His main research interests are hyper- Control Science and Engineering from Harbin Institute of Technology
spectral image denoising and classification. (HIT), in 1985, 1988, and 1995, respectively. He has been a professor
with the Department of Control Science and Engineering, HIT, since
Qiang Wang received his BS, MS, and PhD degrees in the 1997. His current research interests include signal and image
Department of Control Science and Engineering from Harbin Institute processing, remote sensing, fault diagnosis, and digital ultrasonic
of Technology (HIT), in 1998, 2000, and 2004, respectively. He has imaging.