PSC 128/NPB 163

Lab #6 - supervised learning

(due Friday, 2/13)

1. Apples vs. oranges (simple)

In this problem, you will train a network to separate apples from oranges. The files apples.mat and oranges.mat contain two-dimensional data from two different spectral measurements of these fruit. There are ten examples of each.

a) Plot the apple/orange data, using a ‘+’ for apples and an ‘o’ for oranges. Is the data linearly separable?

b) Train a linear neuron (ADALINE) to learn the discrimination by assigning apples to +1 and oranges to -1. Make a plot of the error as a function of trial number, and show the final solution (i.e., the separating hyperplane) superimposed in the same plot with your data.

c) Now put a sigmoidal output nonlinearity on your neuron and train it to assign apples a value of 1 and oranges a value of 0. Again plot the error as a function of trial number, and show the final solution together with your data. How does the final error differ from that in (b)? Why?

You can use the script lab61.m on the class web page to get you started with this problem.


2. Apples vs. oranges (complex)

Now we introduce a new variety of apple and orange, but we still want to successfully separate all of the apples from all of the oranges. The new data are in the files apples2.mat and oranges2.mat. Again, there are ten examples of each.

a) Plot the new apple/orange data together with the old data. Is the problem still linearly separable?

b) Train a two-stage network (using backprop) to learn the discrimination. Plot the error as a function of trial number. Show the final solution by plotting the separating hyperplanes in both the input space and in the hidden unit space (and show the data transformed into the hidden space as well).

You can use the script lab62.m on the class web page for this assignment. The script lab62m.m uses a momentum term and may have better luck avoiding local minima.