1. Two-dimensional data.

You must download the file data2d.mat for this problem. It contains two arrays, D1 and D2, each of which contains 1000 data points in two dimensions.

a) Plot the two-dimensional data in the array D1 (as points, not lines!).

b) Now train a single linear neuron on this data using Hebbian learning (unconstrained). Plot the weight vector along with the data on each weight update.

c) Now apply Oja's single-neuron learning rule to constrain the growth of the weight vector, and again show how the weight vector evolves during learning.

d) Use Sanger's rule to train two neurons to represent the principal components of the data.

e) Now apply steps a-d to the data in array D2.
 
 

2. Faces.

You must download the file faces2.mat for this problem. It is a 3840x48 array. Each column contains the image of a face. You must reshape it to 60x64 to make it an image. For example, to look at all the faces you would type

colormap gray
for i=1:48
  imagesc(reshape(faces2(:,i),60,64)), axis image
  drawnow
  pause(0.5)
end

b) Use Sanger's rule to learn the first 2 principal components of the data (so-called "eigenfaces"). Show what these look like.

c) Plot each face as a point in the two-dimensional space spanned by the first two PC's.  Use a different color marker for each face (note that there are 16 faces in 3 poses each).

d) Show what reconstructions of each face look like using the first two PC's only.