# Comparison of PCA, NNMF, and ICA over image de-noising

## Introduction

In a previous blog post, , I compared Principal Component Analysis (PCA) / Singular Value Decomposition (SVD) and Non-Negative Matrix Factorization (NNMF) over a collection of noised images of digit handwriting from the MNIST data set, , which is available in Mathematica.

This blog post adds to that comparison the use of Independent Component Analysis (ICA) proclaimed in my previous blog post, .

## Computations

The ICA related additional computations to those in  follow.

First we load the package IndependentComponentAnalysis.m :

``````Import["https://raw.githubusercontent.com/antononcube/\
MathematicaForPrediction/master/IndependentComponentAnalysis.m"]``````

From that package we can use the function `IndependentComponentAnalysis` to find components:

``````{S, A} = IndependentComponentAnalysis[Transpose[noisyTestImagesMat], 9, PrecisionGoal -> 4.5];
Norm[noisyTestImagesMat - Transpose[S.A]]/Norm[noisyTestImagesMat]
(* 0.592739 *)``````

Let us visualize the norms of the mixing matrix A :

``````norms = Norm /@ A;
ListPlot[norms, PlotRange -> All, PlotLabel -> "Norms of A rows",
PlotTheme -> "Detailed"] //
ColorPlotOutliers[TopOutliers@*HampelIdentifierParameters]
pos = OutlierPosition[norms, TopOutliers@*HampelIdentifierParameters]`````` Next we can visualize the found "source" images:

``````ncol = 2;
Grid[Partition[Join[
MapIndexed[{#2[], Norm[#],
Transpose[S]] /. (# -> Style[#, Red] & /@ pos),
Table["", {ncol - QuotientRemainder[Dimensions[S][], ncol][]}]
], ncol], Dividers -> All]`````` After selecting several of source images we zero the rest by modifying the matrix A:

``````pos = {6, 7, 9};
norms = Norm /@ A;
dA = DiagonalMatrix[
ReplacePart[ConstantArray[0, Length[norms]], Map[List, pos] -> 1]];
newMatICA =
Transpose[Map[# + Mean[Transpose[noisyTestImagesMat]] &, S.dA.A]];
denoisedImagesICA = Map[Image[Partition[#, dims[]]] &, newMatICA];``````

## Visual comparison of de-noised images

Next we visualize the ICA de-noised images together with the original images and the SVD and NNMF de-noised ones.

There are several ways to present that combination of images.   ## Comparison using classifiers

We can further compare the de-noising results by building digit classifiers and running them over the de-noised images.

``````icaCM = ClassifierMeasurements[digitCF,
denoisedImagesICA) -> testImageLabels]]``````

We can see that with ICA we get better results than with PCA/SVD, probably not as good as NNMF, but very close. ## All images of this blog post A. Antonov, "Comparison of PCA and NNMF over image de-noising", (2016), MathematicaForPrediction at WordPress.

 A. Antonov, "Independent Component Analysis for multidimensional signals", (2016), MathematicaForPrediction at WordPress.

 Wikipedia entry, MNIST database.

## 2 thoughts on “Comparison of PCA, NNMF, and ICA over image de-noising”

This site uses Akismet to reduce spam. Learn how your comment data is processed.