Activity 19: Restoration of Blurred Image

"... Twilight on the frozen lake
North wind about to break
On footprints in the snow
Silence down below."

This activity initially distorts an image through blurring and adding of gaussian noise. It aims to restore the distorted image based on previous knowledge on the nature of distortions introduced.

Say we initially have an input image f(x,y) (refer to the image below) introduced by a degradation function h(x,y) and Gaussian noise g(x,y), restoration filters then act on the distorted image to produce an image close to the original.

The restored image can be represented by the following equation in spatial domain

and by the following equation in frequency domain.

where H in fourier space is given by the following formula. I used a=0.0005 (vertical blur) and b=0.05 (horizontal blur). The more I increase a and b values, the more blurry the image becomes. Hence, the choice of a and b values correspond to a somewhat blurred image with poor image visibility.

The original image as shown below represents the function f(x,y) and F(u,v) in real space and fourier space respectively.

Original Image

(http://www.musicroom.fr/images/catalogue/fullsize/OP52811.jpg)

Blurring (left) and adding gaussian noise (right) is mathematically represented by the equations presented above.

Left: Blurred Image, Right: Blur+Gaussian Noise

Details on the applied distortions are known beforehand. Restoration is then possible though utilizing functions to obtain the target restored image. Restoration comes in two forms, one is represented by the equation below (6) and the other one is given by equation 7.

After several attempts of trial and error, the best suited K value was 0.03. Surprisingly, the ratio between Sn and Sf is 25.194144. By virtue of comparison, K is equal to this ratio.

And now we obtain the following restored images. It can be oberved that using the ration between Sn and Sf for each pixel in the image turned out to be a better restoration option than K estimates.

Left: Restored (eqn 6), Right: Restored (eqn 7)

Equation 6 is better than equation 7 since the latter only estimates a fixed value for K.


I give myself a grade of 10 for this activity. I think I have understood all concepts here.
Thanks to Mandee and Carmen for the very interesting discussions.


And since this is my last blog for this course, Thanks Ma'am Jing and Ma'am Gay for the patience, and brilliant mentoring. :D


Lyrics: Portishead, Bob Dylan

Activity 18: Noise Models and Basic Image Restorations

"You're gonna cry for pills

And your head's gonna be in a knot,
But the pills are gonna cost more
Than what you've got."

This activity aims to create distorted images by adding different types of noises to them namely, Gaussian, Exponential, Rayleigh, Salt and Pepper, Erling and Normal noise. After which, the distorted images are restored through also different types of filters namely: Arithmetic, Contraharmonic positive, Contaharmonic negative, Exponential, Geometric, Harmonic.

(http://medisinmusicforthemasses.files.wordpress.com/2008/10/terry_bozzio_drumset2.jpg)

Original Image (left), Histogram (right)
sample image(down)

We note that the visual differences are more often than not unnoticeable and thus negligible. Hence, there is a need to plot histograms for each of the images and interpret from them.

SALT AND PEPPER
(Salt and Pepper, Arithmetic, Contraharmonic positive, Contraharmonic negative, Geometric, Harmonic)

All types of filters except geometric filter seem to exhibit the same results.

SALT
(No filter, Arithmetic, Contraharmonic positive, Contaharmonic negative, Geometric, Harmonic)






All filters seem to have similar effect on this type of noise. Based on peaks presented in the histogram, Geometric filtering is the best among other types.


RAYLEIGH
(No filter, Arithmetic, Contraharmonic positive, Contaharmonic negative, Geometric, Harmonic)







Arithmetic filtering works best for this type of noise as seen on the histogram. Peaks are best portrayed in this type of filtering method.



PEPPER
(No filter, Arithmetic, Contraharmonic positive, Contaharmonic negative, Geometric, Harmonic)








Nothing seems to work for the Pepper noise.


NORMAL
(No filter, Arithmetic, Contraharmonic positive, Contaharmonic negative,Geometric, Harmonic)







Positive Contraharmonic exhibited the best type of filtering method for this type of noise based on peaks of histogram. Geometric on the other hand shows more traces of leftover noise than the rest of the filters used.



GAUSS
(No filter, Arithmetic, Contraharmonic positive, Contaharmonic negative,Geometric, Harmonic)






Positive Contraharmonic is the best type of filtering for this type of noise as shown by the peaks of the histogram. Geometric filter on the other hand has a problem on leftover noise.



EXPONENTIAL
(No filter, Arithmetic, Contraharmonic positive, Contaharmonic negative, Geometric, Harmonic)






Geometric filtering is the best method here based on peaks of histogram.

ERLANG
(No filter, Arithmetic, Contraharmonic positive, Contaharmonic negative, Geometric, Harmonic)

Positive Contraharmonic works best for this type of noise.

Different types of filtering methods work for different types of noise. Hence, it is best to identify the type of noise beforehand and pick the appropriate filtering technique afterward.

Im not quite sure if the results i have obtained are correct. However, I still give myself a grade of 10 for this activity because I think i have understood all concepts.

Thanks to Mandee and Carmen for pruductive discussions. And to Gilbert (of course) for the shortcut on Rayleigh Noise.

Activity 17: Photometric Stereo

"... You're gonna need
You're gonna need my help someday
Well, if you can't quit your shinnin'
Please quit your low down ways."

Photometric Stereo is a technique used by computers to view object surfaces through exposing them to different types of lighting conditions [1]. This activity aims to extract shapes of surfaces illuminated under four types of lighting conditions.

We can estimate shapes of surfaces by taking several pictures of the surface of the image exposed to light source at different locations. Shadows on the images contain the necessaty infromation for surface rescontrcution. In our case, four images of a spherical object exposed under four types of lighting conditions brought upon by four different locations the light source is located which are imbedded in a matlab file were initially extracted. Images of I1, I2, I3 and I4 are respectively shown below.

I1, I2

I3, I4

Which leads to the construction of a matrix of locations of the following form:

V ( already provided) is equal to the following matrix in this activity

0.085832 0.17365 0.98106 0.085832 - 0.17365 0.98106 0.17365 0. 0.98481 0.16318 - 0.34202 0.92542

Each row corresponds to the N number light sources while columns indicate correspodning x,y and z coordinates. I and V are already provided. We can now compute for g:

and solve for the normal vector n:

To get the shape of the sphere from the normal, the elevation z whcih is also a function of x and y is computed through the following formula

where

The final 3d constrcution obtained is shown below.

Reference:
[1] http://en.wikipedia.org/wiki/Photometric_Stereo

Activity 16: Neural Networks

".. The sweet pretty things are in bed now of course The city fathers they're trying to endorse"

Numerous processes are known to classify objects among classes they belong to. Linear Discriminant Analysis was already implemented in the previous experiment. This acivity introduces yet another effective classifier known as a neural network. It is a computational model of how neurons in the brain work. The latter method is known more effective than LDA since it is independent of recognition rules to perform classification. The neural network is initially fed with information through which it learns from these set of examples. Once a new set of information is tested on the neural network, it then readily classifies these elements based on previously acquired knowledge.

Again, we use the same sets of data values used in the previous activities. Through the code provided by Cole, a student of AP 186 a year ago, computed class was approximated and was found to perfectly correspond to the actual classes the elements belong to. Thus, we can now say that the network has indeed learned from the information fed to it. It is now ready to classify the second half of the data set (test set).

Following the same procedures as the training set, we get the following results as summarized in the table below.

Once again, predictions were found to be 100% accurate for all elements in the set.

Rating: 10
I have understood all concepts in this activity.

Activity15: Probabilistic Classification

"...If the wine don't come to the top of yer cup
If the wind's got you sideways with with one hand holdin' on
And the other starts slipping and the feeling is gone"

This activity used similar samples as in the previous activity (sample group 1: Pillows which are wafer covered chocolate and sample group 2: Cheezit which are cheese flavored chips). Classification among the two major classes was already done in the previous exercise. However, a problem arises if a classifier wrongly groups certain samples into their appropriate classes. The aim here is to minimize the risk or loss in classification through a statistical method known as Linear Discrimant Analysis (LDA).

LDA is a method used in Statistics to find linear combination of features or characteristics which best separate two or more classes of objects or events [1]. In other words, LDA is an effective classifier of objects based on features they exhibit.

Now we start with a training set composed of 4 Pillows assigned class 1 and 4 Cheezit assigned class 2. Following all steps provided by the LDA tutorial in [2] and implementing them in Scilab, we come up with values together with discriminant functions shown below.

Discriminant functions (f1 and f2) take full responsibility for classifying between classes Pillows (1) and Cheezit (2). For each element in the set, a comparison is made between the two functions wherein the larger valued function takes over (e.g. if f1>f2 means object belongs to class 1). Results show that LDA exhibited 0% classification error. Indeed, the first 4 elements highlighted in brown were precisely classified belonging to the class Pillows and correspondingly the last four elements highlighted in yellow as Cheezit.

Now we use the same method for the test samples. Results are shown below.

All of the samples in the test set were classified correctly. LDA is thus an effective method for classifying objects to their respective classes.

Reference:
[1] http://en.wikipedia.org/wiki/Linear_discriminant_analysis
[2] http://people.revoledu.com/kardi/tutorial/LDA/Numerical%20Example.html

Rating: 10
I think I have fully understood all concepts discussed here. All elements were classified precisely.

Thanks to mandee and carmen for useful discussions, and the company as well.

Activity 14: Pattern Recognition

"... But I can hear another drum beating for the dead that rise
Whom nature's beast fear as they come and all I see are dark eyes."



A pattern refers to a set of features or characteristics of objects. Humans readily classify objects around them based on common sense and intuition. On the other hand, Computers do not possess this complex ability despite the drastic advancement of technology these days.

This activity aims to classify certain objects through pattern recognition. 12 pieces of pillows and cheezit were classified based on two features: RGB values and Area. Six elements of each class were used as training elements while the rest served as test elements.

Pictures of 12 pieces for each class (Pillows and Cheezit) placed side by side were initially taken.

Pillows vs Cheezit



AREA ESTIMATION

For area computation, images of test sets (6 elements for each class)were thresholded in Scilab resulting to images shown below (left: Pillows; right: Cheezit).

Similar methods used in Activity 9 were implemented on these images. The function bwlabel in scilab was used to label pixels which correspond to a specific element (piece of Pillow or Cheezit) in the image. After which, pixels with the same labels were added. This sum corresponds to the estimated area of a single element. The same methods were implemented on the other half (test set).

The table below summarizes area of samples 1-6 (training set) and 7-12 (test set).

ESTIMATED AREA


Pillows
(Training Set)
Mean: 7424.1667
Std dev: 501.19115

Cheezit
(Training Set)
Mean: 16737.167
Std dev: 1413.6567



Red-Green-Blue

RGB values of each of the elements for both classes (training and test set) were sampled through image patches and summarized in the table shown below.

RGB VALUES


Pillows
(MEAN)
R: 0.514855633
G: 0.3820315
B: 0.243684817

Cheezit
(MEAN)
R: 0.514855633
G: 0.3820315
B: 0.243684817

CLASSIFICATION PROPER

RGB values and Area estimates were then plotted for all test elements. including mean values of training elements. The yellow data point corresponds to the mean value for Cheezit while the brown one corresponds to Pillows. Based on the figures shown, we infer that an imaginary line drawn along the diagonal would separate the two classes into Pillows and Cheezit. Clearly, the blue data points here correspond to Cheezit while the red ones to Pillows.

Rating: 10
I have understood all concepts and did what was asked in this activity.

Thanks to mandee and carmen for the pictures (and snack).

Activity 13: Correcting Geometric Distortions

"... Blue moon, you saw me standing alone
Without a dream in my heart, without a love of my own."

This activity aims to correct distortions on images. Among the most common distortion is barrel distortion where the image is often viewed somewhat distorted in mid areas and pinched on its sides.

There are many ways to restore distorted images, among others is the interpolation technique categorized into three groups namely: Nearest neighbor, Cubic convolution and Bilinear. The last two techniques were compared in this activity and we neglect the first technique since it renders low quality images.

Original Image

Starting with four corner points of the best rectangle in the distorted image, an ideal image can then be generated.

Arrows connect corresponding vertex points of ideal (right image) and distorted (left image). These sets of corner points for both ideal and distorted images were collected with the use of the function locate in Scilab.

Point Locations in x and y coordinates

The above matrices were used to numerically relate ideal and distorted coordinates. This relationship is best described by constants c defined by equations below.

Equations shown below represent locations for the distorted image in terms of x and y coordinates in the ideal image.

Grayscale values corresponding to the computed x and y coordinates are immediately copied into a blank pixel in the ideal grid if locations are integer-values. Interpolation techniques are performed otherwise.

Nearest Neighbor Interpoation technique oly refers to rounding off of non-integer values to nearest values (hence nearest neighbor) and assume that these locations correspond to the correct grayscale pixel values ready to be copied. Bilinear interpolation technique on the other hand requires a more rigorous approach than the former technique since it computes the pixel value based on four nearest pixel values, hence the equation shown.

Results shown below have proven that Bilinear interpolation technique renders a clearer thus better image than Nearest Neighbor Technique.

Nearest Neighbor Interpolation Technique


Bilinear Interpolation Technique

I give myself a grade of 10 for completing the task and understanding the concepts quite well in this activity.

Thanks to Mandee and Carmen for very useful discussions regarding misconceptions on the problem and the code itself.