https://rctn.org/w/index.php?title=VS298_(Fall_06):_Suggested_projects&feed=atom&action=historyVS298 (Fall 06): Suggested projects - Revision history2024-03-29T14:44:25ZRevision history for this page on the wikiMediaWiki 1.39.4https://rctn.org/w/index.php?title=VS298_(Fall_06):_Suggested_projects&diff=2664&oldid=prevBruno at 02:48, 31 October 20062006-10-31T02:48:58Z<p></p>
<table style="background-color: #fff; color: #202122;" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 02:40, 31 October 2006</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l19">Line 19:</td>
<td colspan="2" class="diff-lineno">Line 19:</td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Analysis of Hopfield dynamics.''' The Hopfield dynamics may be seen as performing a form of gradient descent on the energy function - i.e., the state of each unit, <math>V_i</math>, follows a monotonically increasing function of the gradient rather than the gradient itself. Is the resulting trajectory more efficient for reaching the energy minimum than what you would get from doing steepest descent?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Analysis of Hopfield dynamics.''' The Hopfield dynamics may be seen as performing a form of gradient descent on the energy function - i.e., the state of each unit, <math>V_i</math>, follows a monotonically increasing function of the gradient rather than the gradient itself. Is the resulting trajectory more efficient for reaching the energy minimum than what you would get from doing steepest descent?</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* '''Bump circuits.''' Implement Kechen Zhang's bump circuit model. How robust is the model to perturbations of the weights? How might such a circuit to self-correct for any imperfections in the weights?</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* '''Bump circuits.''' Implement Kechen Zhang's bump circuit model <ins style="font-weight: bold; text-decoration: none;">discussed in class</ins>. How robust is the model to perturbations of the weights? How might such a circuit to self-correct for any imperfections in the weights?</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Map-seeking circuits.''' David Arathorn has described a neural circuit for doing invariant object recognition which utilizes three-way interactions among units - see "Map-seeking circuits in Visual Cognition," Stanford University Press, 2002. However, his implementation of the associative memory uses grandmother cells. Try using instead a distributed representation for the memory using a Hopfield network. </div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Map-seeking circuits.''' David Arathorn has described a neural circuit for doing invariant object recognition which utilizes three-way interactions among units - see "Map-seeking circuits in Visual Cognition," Stanford University Press, 2002. However, his implementation of the associative memory uses grandmother cells. Try using instead a distributed representation for the memory using a Hopfield network. </div></td></tr>
</table>Brunohttps://rctn.org/w/index.php?title=VS298_(Fall_06):_Suggested_projects&diff=2663&oldid=prevBruno at 02:48, 31 October 20062006-10-31T02:48:01Z<p></p>
<table style="background-color: #fff; color: #202122;" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 02:40, 31 October 2006</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l17">Line 17:</td>
<td colspan="2" class="diff-lineno">Line 17:</td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Hardware implementation of associative memory.''' The analog Hopfield model has a direct physical implementation as an electrical circuit of resistors, capacitors, and op-amps. Try building a scaled-down version of this model in hardware. What issues arise in the implementation of this model? How long does it take to converge to a local minimum?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Hardware implementation of associative memory.''' The analog Hopfield model has a direct physical implementation as an electrical circuit of resistors, capacitors, and op-amps. Try building a scaled-down version of this model in hardware. What issues arise in the implementation of this model? How long does it take to converge to a local minimum?</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* '''Analysis of Hopfield dynamics.''' The Hopfield dynamics may be seen as performing a form of gradient descent on the energy function - i.e., the state of each unit, <math>V_i</math>, follows a monotonically increasing function of the gradient. Is the resulting trajectory more efficient for reaching the energy minimum than what you would get from doing steepest descent?</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* '''Analysis of Hopfield dynamics.''' The Hopfield dynamics may be seen as performing a form of gradient descent on the energy function - i.e., the state of each unit, <math>V_i</math>, follows a monotonically increasing function of the gradient <ins style="font-weight: bold; text-decoration: none;">rather than the gradient itself</ins>. Is the resulting trajectory more efficient for reaching the energy minimum than what you would get from doing steepest descent?</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Bump circuits.''' Implement Kechen Zhang's bump circuit model. How robust is the model to perturbations of the weights? How might such a circuit to self-correct for any imperfections in the weights?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Bump circuits.''' Implement Kechen Zhang's bump circuit model. How robust is the model to perturbations of the weights? How might such a circuit to self-correct for any imperfections in the weights?</div></td></tr>
</table>Brunohttps://rctn.org/w/index.php?title=VS298_(Fall_06):_Suggested_projects&diff=2662&oldid=prevBruno at 02:47, 31 October 20062006-10-31T02:47:08Z<p></p>
<table style="background-color: #fff; color: #202122;" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 02:39, 31 October 2006</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l17">Line 17:</td>
<td colspan="2" class="diff-lineno">Line 17:</td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Hardware implementation of associative memory.''' The analog Hopfield model has a direct physical implementation as an electrical circuit of resistors, capacitors, and op-amps. Try building a scaled-down version of this model in hardware. What issues arise in the implementation of this model? How long does it take to converge to a local minimum?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Hardware implementation of associative memory.''' The analog Hopfield model has a direct physical implementation as an electrical circuit of resistors, capacitors, and op-amps. Try building a scaled-down version of this model in hardware. What issues arise in the implementation of this model? How long does it take to converge to a local minimum?</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* '''Analysis of Hopfield dynamics.''' The Hopfield dynamics may be seen as performing a form of gradient descent on the energy function - i.e., the state of each unit, <del style="font-weight: bold; text-decoration: none;">$</del>V_i<del style="font-weight: bold; text-decoration: none;">$</del>, follows a monotonically increasing function of the gradient. Is the resulting trajectory more efficient for reaching the energy minimum than what you would get from doing steepest descent?</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* '''Analysis of Hopfield dynamics.''' The Hopfield dynamics may be seen as performing a form of gradient descent on the energy function - i.e., the state of each unit, <ins style="font-weight: bold; text-decoration: none;"><math></ins>V_i<ins style="font-weight: bold; text-decoration: none;"></math></ins>, follows a monotonically increasing function of the gradient. Is the resulting trajectory more efficient for reaching the energy minimum than what you would get from doing steepest descent?</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Bump circuits.''' Implement Kechen Zhang's bump circuit model. How robust is the model to perturbations of the weights? How might such a circuit to self-correct for any imperfections in the weights?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Bump circuits.''' Implement Kechen Zhang's bump circuit model. How robust is the model to perturbations of the weights? How might such a circuit to self-correct for any imperfections in the weights?</div></td></tr>
</table>Brunohttps://rctn.org/w/index.php?title=VS298_(Fall_06):_Suggested_projects&diff=2661&oldid=prevBruno at 02:46, 31 October 20062006-10-31T02:46:21Z<p></p>
<table style="background-color: #fff; color: #202122;" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 02:38, 31 October 2006</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l7">Line 7:</td>
<td colspan="2" class="diff-lineno">Line 7:</td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Sparse coding and decorrelation.''' Implement [http://redwood.berkeley.edu/~amir/vs298/foldiak90.pdf Peter Foldiak's network] and train it on the handwritten digits above to learn the features of this data. You may wish to then try supervised learning on the learned features to see if it has simplified the classification problem.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Sparse coding and decorrelation.''' Implement [http://redwood.berkeley.edu/~amir/vs298/foldiak90.pdf Peter Foldiak's network] and train it on the handwritten digits above to learn the features of this data. You may wish to then try supervised learning on the learned features to see if it has simplified the classification problem.</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* '''Cortical maps.''' The elastic net model of [http://redwood.berkeley.edu/~amir/vs298/durbin-mitchison.pdf Durbin and Mitchison] is typical of many cortical map models in that they learn directly on a parameterized feature space. But the cortex simply gets a bunch of inputs from the LGN, and so it needs to learn features such as orientation at the same time as it organizes them into a feature map. How would you go about learning a feature map for orientation position directly from simulated LGN inputs? (You may wish to consult the book of [http://nn.cs.utexas.edu/computationalmaps/ Risto Miikkulainen] for recent efforts in this area.)</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* '''Cortical maps.''' The elastic net model of [http://redwood.berkeley.edu/~amir/vs298/durbin-mitchison.pdf Durbin and Mitchison] is typical of many cortical map models in that they learn directly on a parameterized feature space. But the cortex simply gets a bunch of inputs from the LGN, and so it needs to learn features such as orientation at the same time as it organizes them into a feature map. How would you go about learning a feature map for orientation <ins style="font-weight: bold; text-decoration: none;">and </ins>position directly from simulated LGN inputs? (You may wish to consult the book of [http://nn.cs.utexas.edu/computationalmaps/ Risto Miikkulainen] for recent efforts in this area.)</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''The 'magic TV'''' Let's say you wake up one day to find someone rewired your optic nerve (or you have been implanted with a prosthetic retina). The signals from retina to brain are still intact, but the wires are all mixed up in the wrong place. Since neighboring pixels in natural images are correlated, it should be possible to learn a remapping that "descrambles" the image by exploiting these correlations. See if you can train a Kohonen-style network to learn the proper topographic mapping of an image based on the statistics of natural images. (Kohonen dubbed this problem 'the Magic TV'.)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''The 'magic TV'''' Let's say you wake up one day to find someone rewired your optic nerve (or you have been implanted with a prosthetic retina). The signals from retina to brain are still intact, but the wires are all mixed up in the wrong place. Since neighboring pixels in natural images are correlated, it should be possible to learn a remapping that "descrambles" the image by exploiting these correlations. See if you can train a Kohonen-style network to learn the proper topographic mapping of an image based on the statistics of natural images. (Kohonen dubbed this problem 'the Magic TV'.)</div></td></tr>
<tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l17">Line 17:</td>
<td colspan="2" class="diff-lineno">Line 17:</td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Hardware implementation of associative memory.''' The analog Hopfield model has a direct physical implementation as an electrical circuit of resistors, capacitors, and op-amps. Try building a scaled-down version of this model in hardware. What issues arise in the implementation of this model? How long does it take to converge to a local minimum?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Hardware implementation of associative memory.''' The analog Hopfield model has a direct physical implementation as an electrical circuit of resistors, capacitors, and op-amps. Try building a scaled-down version of this model in hardware. What issues arise in the implementation of this model? How long does it take to converge to a local minimum?</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* '''Analysis of Hopfield dynamics.''' The Hopfield dynamics may be seen as performing a form of gradient descent on the energy function. Is the trajectory <del style="font-weight: bold; text-decoration: none;">to </del>the energy minimum <del style="font-weight: bold; text-decoration: none;">more efficient </del>than what you would get from doing steepest descent?</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* '''Analysis of Hopfield dynamics.''' The Hopfield dynamics may be seen as performing a form of gradient descent on the energy function <ins style="font-weight: bold; text-decoration: none;">- i.e., the state of each unit, $V_i$, follows a monotonically increasing function of the gradient</ins>. Is the <ins style="font-weight: bold; text-decoration: none;">resulting </ins>trajectory <ins style="font-weight: bold; text-decoration: none;">more efficient for reaching </ins>the energy minimum than what you would get from doing steepest descent?</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* '''Bump circuits.''' Implement Kechen Zhang's bump circuit model. How robust is the model to perturbations of the weights?</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* '''Bump circuits.''' Implement Kechen Zhang's bump circuit model. How robust is the model to perturbations of <ins style="font-weight: bold; text-decoration: none;">the weights? How might such a circuit to self-correct for any imperfections in </ins>the weights?</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* '''<del style="font-weight: bold; text-decoration: none;">Arathorn's map</del>-seeking <del style="font-weight: bold; text-decoration: none;">circuit</del>.''' David Arathorn has described a neural circuit for doing invariant object recognition which utilizes three-way interactions among units. However, his implementation of the associative memory uses grandmother cells. Try using instead a distributed representation for the memory using a Hopfield network. </div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* '''<ins style="font-weight: bold; text-decoration: none;">Map</ins>-seeking <ins style="font-weight: bold; text-decoration: none;">circuits</ins>.''' David Arathorn has described a neural circuit for doing invariant object recognition which utilizes three-way interactions among units <ins style="font-weight: bold; text-decoration: none;">- see "Map-seeking circuits in Visual Cognition," Stanford University Press, 2002</ins>. However, his implementation of the associative memory uses grandmother cells. Try using instead a distributed representation for the memory using a Hopfield network. </div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Entropy of natural images.''' Because of the strong statistical dependencies that exist among pixels in natural images, the true entropy of an image patch is far less than N x (entropy/pixel), where N is the number of pixels. But calculating the true entropy is impossible in practice because it requires collecting the full joint pdf over an image patch, and for any reasonably sized patch this is intractable. Thus it is currently unknown. But there are other ways to estimate the entropy of data beyond direct caculation from the pdf. See for example the recent work of [mailto:djf3@cornell.edu David Field] and colleagues, or methods based on the Hausdorff dimension. Implement one of these methods, or see if you can come up with your own method, for estimating the true entropy of images.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Entropy of natural images.''' Because of the strong statistical dependencies that exist among pixels in natural images, the true entropy of an image patch is far less than N x (entropy/pixel), where N is the number of pixels. But calculating the true entropy is impossible in practice because it requires collecting the full joint pdf over an image patch, and for any reasonably sized patch this is intractable. Thus it is currently unknown. But there are other ways to estimate the entropy of data beyond direct caculation from the pdf. See for example the recent work of [mailto:djf3@cornell.edu David Field] and colleagues, or methods based on the Hausdorff dimension. Implement one of these methods, or see if you can come up with your own method, for estimating the true entropy of images.</div></td></tr>
</table>Brunohttps://rctn.org/w/index.php?title=VS298_(Fall_06):_Suggested_projects&diff=2645&oldid=prevBruno at 05:58, 28 October 20062006-10-28T05:58:43Z<p></p>
<table style="background-color: #fff; color: #202122;" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 05:50, 28 October 2006</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l14">Line 14:</td>
<td colspan="2" class="diff-lineno">Line 14:</td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Sparse codes and associative memory.''' The advantage of storing and recalling patterns using an associative memory as opposed to a conventional computer memory is 1) parallel search, and 2) denoising (recall of an uncorrupted pattern from partial or degraded input). However, associative memory models do not work well with natural data such as images or sound directly. Rather, they are best suited (have highest capacity) for sparse patterns (i.e., patterns with many zeros). Recent work (to be discussed in class) has shown how it is possible to convert natural images and sounds into a sparse format, and there is some evidence for this happening in the brain. See if you can link these ideas in order to store natural images or sounds in an associative memory. </div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Sparse codes and associative memory.''' The advantage of storing and recalling patterns using an associative memory as opposed to a conventional computer memory is 1) parallel search, and 2) denoising (recall of an uncorrupted pattern from partial or degraded input). However, associative memory models do not work well with natural data such as images or sound directly. Rather, they are best suited (have highest capacity) for sparse patterns (i.e., patterns with many zeros). Recent work (to be discussed in class) has shown how it is possible to convert natural images and sounds into a sparse format, and there is some evidence for this happening in the brain. See if you can link these ideas in order to store natural images or sounds in an associative memory. </div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">* '''Hardware implementation of associative memory.''' The analog Hopfield model has a direct physical implementation as an electrical circuit of resistors, capacitors, and op-amps. Try building a scaled-down version of this model in hardware. What issues arise in the implementation of this model? How long does it take to converge to a local minimum?</ins></div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">* '''Analysis of Hopfield dynamics.''' The Hopfield dynamics may be seen as performing a form of gradient descent on the energy function. Is the trajectory to the energy minimum more efficient than what you would get from doing steepest descent?</ins></div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">* '''Bump circuits.''' Implement Kechen Zhang's bump circuit model. How robust is the model to perturbations of the weights?</ins></div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">* '''Arathorn's map-seeking circuit.''' David Arathorn has described a neural circuit for doing invariant object recognition which utilizes three-way interactions among units. However, his implementation of the associative memory uses grandmother cells. Try using instead a distributed representation for the memory using a Hopfield network. </ins></div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Entropy of natural images.''' Because of the strong statistical dependencies that exist among pixels in natural images, the true entropy of an image patch is far less than N x (entropy/pixel), where N is the number of pixels. But calculating the true entropy is impossible in practice because it requires collecting the full joint pdf over an image patch, and for any reasonably sized patch this is intractable. Thus it is currently unknown. But there are other ways to estimate the entropy of data beyond direct caculation from the pdf. See for example the recent work of [mailto:djf3@cornell.edu David Field] and colleagues, or methods based on the Hausdorff dimension. Implement one of these methods, or see if you can come up with your own method, for estimating the true entropy of images.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Entropy of natural images.''' Because of the strong statistical dependencies that exist among pixels in natural images, the true entropy of an image patch is far less than N x (entropy/pixel), where N is the number of pixels. But calculating the true entropy is impossible in practice because it requires collecting the full joint pdf over an image patch, and for any reasonably sized patch this is intractable. Thus it is currently unknown. But there are other ways to estimate the entropy of data beyond direct caculation from the pdf. See for example the recent work of [mailto:djf3@cornell.edu David Field] and colleagues, or methods based on the Hausdorff dimension. Implement one of these methods, or see if you can come up with your own method, for estimating the true entropy of images.</div></td></tr>
</table>Brunohttps://rctn.org/w/index.php?title=VS298_(Fall_06):_Suggested_projects&diff=2625&oldid=prevBruno at 05:06, 23 October 20062006-10-23T05:06:13Z<p></p>
<table style="background-color: #fff; color: #202122;" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 04:58, 23 October 2006</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l17">Line 17:</td>
<td colspan="2" class="diff-lineno">Line 17:</td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Entropy of natural images.''' Because of the strong statistical dependencies that exist among pixels in natural images, the true entropy of an image patch is far less than N x (entropy/pixel), where N is the number of pixels. But calculating the true entropy is impossible in practice because it requires collecting the full joint pdf over an image patch, and for any reasonably sized patch this is intractable. Thus it is currently unknown. But there are other ways to estimate the entropy of data beyond direct caculation from the pdf. See for example the recent work of [mailto:djf3@cornell.edu David Field] and colleagues, or methods based on the Hausdorff dimension. Implement one of these methods, or see if you can come up with your own method, for estimating the true entropy of images.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Entropy of natural images.''' Because of the strong statistical dependencies that exist among pixels in natural images, the true entropy of an image patch is far less than N x (entropy/pixel), where N is the number of pixels. But calculating the true entropy is impossible in practice because it requires collecting the full joint pdf over an image patch, and for any reasonably sized patch this is intractable. Thus it is currently unknown. But there are other ways to estimate the entropy of data beyond direct caculation from the pdf. See for example the recent work of [mailto:djf3@cornell.edu David Field] and colleagues, or methods based on the Hausdorff dimension. Implement one of these methods, or see if you can come up with your own method, for estimating the true entropy of images.</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* '''ICA applied to EEG.''' ICA has been successfully applied to EEG signals, to separate noise artifacts from brain signals, and also to find separate signal sources within the brain. See if you can get some EEG data from one of the labs on campus (e.g., Klein or Knight labs) and use ICA to reveal the hidden structure in this data. To get started, look at Makeig S, Bell AJ, Jung T-P, Sejnowski TJ (1996) "Independent Component Analysis of Electroencephalographic Data.<del style="font-weight: bold; text-decoration: none;">'' </del>In: Advances in Neural Information Processing Systems 8, 145-151.</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* '''ICA applied to EEG.''' ICA has been successfully applied to EEG signals, to separate noise artifacts from brain signals, and also to find separate signal sources within the brain. See if you can get some EEG data from one of the labs on campus (e.g., Klein or Knight labs) and use ICA to reveal the hidden structure in this data. To get started, look at Makeig S, Bell AJ, Jung T-P, Sejnowski TJ (1996) "Independent Component Analysis of Electroencephalographic Data.<ins style="font-weight: bold; text-decoration: none;">" </ins>In: Advances in Neural Information Processing Systems 8, 145-151.</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Hierarchical restricted Boltzmann machines.''' [http://redwood.berkeley.edu/~amir/vs298/hinton06.pdf Geoff Hinton] has recently developed a hierarchical auto-encoder network, based on the restricted Boltzmann machine, for modelling structure in data. Implement this network and train on the handwritten digits data or some other dataset of your choosing.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Hierarchical restricted Boltzmann machines.''' [http://redwood.berkeley.edu/~amir/vs298/hinton06.pdf Geoff Hinton] has recently developed a hierarchical auto-encoder network, based on the restricted Boltzmann machine, for modelling structure in data. Implement this network and train on the handwritten digits data or some other dataset of your choosing.</div></td></tr>
</table>Brunohttps://rctn.org/w/index.php?title=VS298_(Fall_06):_Suggested_projects&diff=2624&oldid=prevBruno at 05:05, 23 October 20062006-10-23T05:05:18Z<p></p>
<table style="background-color: #fff; color: #202122;" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 04:57, 23 October 2006</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l17">Line 17:</td>
<td colspan="2" class="diff-lineno">Line 17:</td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Entropy of natural images.''' Because of the strong statistical dependencies that exist among pixels in natural images, the true entropy of an image patch is far less than N x (entropy/pixel), where N is the number of pixels. But calculating the true entropy is impossible in practice because it requires collecting the full joint pdf over an image patch, and for any reasonably sized patch this is intractable. Thus it is currently unknown. But there are other ways to estimate the entropy of data beyond direct caculation from the pdf. See for example the recent work of [mailto:djf3@cornell.edu David Field] and colleagues, or methods based on the Hausdorff dimension. Implement one of these methods, or see if you can come up with your own method, for estimating the true entropy of images.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Entropy of natural images.''' Because of the strong statistical dependencies that exist among pixels in natural images, the true entropy of an image patch is far less than N x (entropy/pixel), where N is the number of pixels. But calculating the true entropy is impossible in practice because it requires collecting the full joint pdf over an image patch, and for any reasonably sized patch this is intractable. Thus it is currently unknown. But there are other ways to estimate the entropy of data beyond direct caculation from the pdf. See for example the recent work of [mailto:djf3@cornell.edu David Field] and colleagues, or methods based on the Hausdorff dimension. Implement one of these methods, or see if you can come up with your own method, for estimating the true entropy of images.</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* '''ICA applied to EEG.''' ICA has been successfully applied to EEG signals, to separate noise artifacts from brain signals, and also to find separate signal sources within the brain. See if you can get some EEG data from one of the labs on campus (e.g., Klein or Knight labs) and use ICA to reveal the hidden structure in this data. To get started, look at Makeig S, Bell AJ, Jung T-P, Sejnowski TJ (1996) <del style="font-weight: bold; text-decoration: none;">``</del>Independent Component Analysis of Electroencephalographic Data.'' In: Advances in Neural Information Processing Systems 8, 145-151.</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* '''ICA applied to EEG.''' ICA has been successfully applied to EEG signals, to separate noise artifacts from brain signals, and also to find separate signal sources within the brain. See if you can get some EEG data from one of the labs on campus (e.g., Klein or Knight labs) and use ICA to reveal the hidden structure in this data. To get started, look at Makeig S, Bell AJ, Jung T-P, Sejnowski TJ (1996) <ins style="font-weight: bold; text-decoration: none;"> "</ins>Independent Component Analysis of Electroencephalographic Data.'' In: Advances in Neural Information Processing Systems 8, 145-151.</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Hierarchical restricted Boltzmann machines.''' [http://redwood.berkeley.edu/~amir/vs298/hinton06.pdf Geoff Hinton] has recently developed a hierarchical auto-encoder network, based on the restricted Boltzmann machine, for modelling structure in data. Implement this network and train on the handwritten digits data or some other dataset of your choosing.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Hierarchical restricted Boltzmann machines.''' [http://redwood.berkeley.edu/~amir/vs298/hinton06.pdf Geoff Hinton] has recently developed a hierarchical auto-encoder network, based on the restricted Boltzmann machine, for modelling structure in data. Implement this network and train on the handwritten digits data or some other dataset of your choosing.</div></td></tr>
</table>Brunohttps://rctn.org/w/index.php?title=VS298_(Fall_06):_Suggested_projects&diff=2623&oldid=prevBruno at 05:02, 23 October 20062006-10-23T05:02:44Z<p></p>
<table style="background-color: #fff; color: #202122;" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 04:54, 23 October 2006</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l19">Line 19:</td>
<td colspan="2" class="diff-lineno">Line 19:</td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''ICA applied to EEG.''' ICA has been successfully applied to EEG signals, to separate noise artifacts from brain signals, and also to find separate signal sources within the brain. See if you can get some EEG data from one of the labs on campus (e.g., Klein or Knight labs) and use ICA to reveal the hidden structure in this data. To get started, look at Makeig S, Bell AJ, Jung T-P, Sejnowski TJ (1996) ``Independent Component Analysis of Electroencephalographic Data.'' In: Advances in Neural Information Processing Systems 8, 145-151.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''ICA applied to EEG.''' ICA has been successfully applied to EEG signals, to separate noise artifacts from brain signals, and also to find separate signal sources within the brain. See if you can get some EEG data from one of the labs on campus (e.g., Klein or Knight labs) and use ICA to reveal the hidden structure in this data. To get started, look at Makeig S, Bell AJ, Jung T-P, Sejnowski TJ (1996) ``Independent Component Analysis of Electroencephalographic Data.'' In: Advances in Neural Information Processing Systems 8, 145-151.</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* '''Hierarchical restricted Boltzmann machines.''' [http://redwood.berkeley.edu/~amir/vs298/hinton06.pdf Geoff Hinton] has recently developed a hierarchical auto-encoder network, based on the restricted Boltzmann machine, for modelling <del style="font-weight: bold; text-decoration: none;">the </del>structure in data. Implement this network and train on the handwritten digits data or some other dataset of your choosing.</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* '''Hierarchical restricted Boltzmann machines.''' [http://redwood.berkeley.edu/~amir/vs298/hinton06.pdf Geoff Hinton] has recently developed a hierarchical auto-encoder network, based on the restricted Boltzmann machine, for modelling structure in data. Implement this network and train on the handwritten digits data or some other dataset of your choosing.</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* '''Integrate-and-fire model neurons.''' </div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* '''Integrate-and-fire model neurons.''' <ins style="font-weight: bold; text-decoration: none;">The [http://redwood.berkeley.edu/~amir/vs298/int-fire/int-fire.html integrate and fire model] is a simple model for capturing the temporal integration and spiking aspects of real neurons. Using such a simplified model it is possible to begin exploring some interesting questions about sensory coding in neurons. For example, how is it possible to encode a continous, time-varying signal using a population of spiking neurons? (See "Spikes" by Rieke et al., or "Principles of Neural Engineering" by Eliasmith and Anderson for an extended discussion of this issue.) You may also wish to explore the effect of adding more realistic biophysical properties, such as the dependence of threshold on membrane potential (see work by [http://redwood.berkeley.edu/~amir/vs298/azouz-gray00.pdf Gray and Azouz]).</ins></div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Oscillations.''' Oscillations in neural activity are pervasive throughout the brain. What kinds of neural circuits are capable of eliciting oscillating behavior in spiking neurons? How could it be coordinated across large regions of cortex? (ask Fritz Sommer) What role might it play in the processing of information? [http://redwood.berkeley.edu/~amir/vs298/hopfield95.pdf John Hopfield] has suggested that spike timing relative to the phase of an ongoing oscillation could code information. What factors would need to be considered in order to make this idea viable?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Oscillations.''' Oscillations in neural activity are pervasive throughout the brain. What kinds of neural circuits are capable of eliciting oscillating behavior in spiking neurons? How could it be coordinated across large regions of cortex? (ask Fritz Sommer) What role might it play in the processing of information? [http://redwood.berkeley.edu/~amir/vs298/hopfield95.pdf John Hopfield] has suggested that spike timing relative to the phase of an ongoing oscillation could code information. What factors would need to be considered in order to make this idea viable?</div></td></tr>
</table>Brunohttps://rctn.org/w/index.php?title=VS298_(Fall_06):_Suggested_projects&diff=2622&oldid=prevBruno at 04:42, 23 October 20062006-10-23T04:42:51Z<p></p>
<table style="background-color: #fff; color: #202122;" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 04:34, 23 October 2006</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l15">Line 15:</td>
<td colspan="2" class="diff-lineno">Line 15:</td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Sparse codes and associative memory.''' The advantage of storing and recalling patterns using an associative memory as opposed to a conventional computer memory is 1) parallel search, and 2) denoising (recall of an uncorrupted pattern from partial or degraded input). However, associative memory models do not work well with natural data such as images or sound directly. Rather, they are best suited (have highest capacity) for sparse patterns (i.e., patterns with many zeros). Recent work (to be discussed in class) has shown how it is possible to convert natural images and sounds into a sparse format, and there is some evidence for this happening in the brain. See if you can link these ideas in order to store natural images or sounds in an associative memory. </div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Sparse codes and associative memory.''' The advantage of storing and recalling patterns using an associative memory as opposed to a conventional computer memory is 1) parallel search, and 2) denoising (recall of an uncorrupted pattern from partial or degraded input). However, associative memory models do not work well with natural data such as images or sound directly. Rather, they are best suited (have highest capacity) for sparse patterns (i.e., patterns with many zeros). Recent work (to be discussed in class) has shown how it is possible to convert natural images and sounds into a sparse format, and there is some evidence for this happening in the brain. See if you can link these ideas in order to store natural images or sounds in an associative memory. </div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* '''Entropy of natural images.''' Because of the strong statistical dependencies that exist among pixels in natural images, the true entropy of an image patch is far less than N x (entropy/pixel), where N is the number of pixels. But calculating the true entropy is impossible in practice because it requires collecting the full joint pdf over an image patch, and for any reasonably sized patch this is intractable. Thus it is currently unknown. But there are other ways to estimate the entropy of data beyond direct caculation from the pdf. See for example the recent work of [mailto:djf3@cornell.edu David Field] and colleagues, or methods based on the Hausdorff dimension. Implement one of these methods, or see if you can come up with your own method for estimating the true entropy of images.</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* '''Entropy of natural images.''' Because of the strong statistical dependencies that exist among pixels in natural images, the true entropy of an image patch is far less than N x (entropy/pixel), where N is the number of pixels. But calculating the true entropy is impossible in practice because it requires collecting the full joint pdf over an image patch, and for any reasonably sized patch this is intractable. Thus it is currently unknown. But there are other ways to estimate the entropy of data beyond direct caculation from the pdf. See for example the recent work of [mailto:djf3@cornell.edu David Field] and colleagues, or methods based on the Hausdorff dimension. Implement one of these methods, or see if you can come up with your own method<ins style="font-weight: bold; text-decoration: none;">, </ins>for estimating the true entropy of images.</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''ICA applied to EEG.''' ICA has been successfully applied to EEG signals, to separate noise artifacts from brain signals, and also to find separate signal sources within the brain. See if you can get some EEG data from one of the labs on campus (e.g., Klein or Knight labs) and use ICA to reveal the hidden structure in this data. To get started, look at Makeig S, Bell AJ, Jung T-P, Sejnowski TJ (1996) ``Independent Component Analysis of Electroencephalographic Data.'' In: Advances in Neural Information Processing Systems 8, 145-151.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''ICA applied to EEG.''' ICA has been successfully applied to EEG signals, to separate noise artifacts from brain signals, and also to find separate signal sources within the brain. See if you can get some EEG data from one of the labs on campus (e.g., Klein or Knight labs) and use ICA to reveal the hidden structure in this data. To get started, look at Makeig S, Bell AJ, Jung T-P, Sejnowski TJ (1996) ``Independent Component Analysis of Electroencephalographic Data.'' In: Advances in Neural Information Processing Systems 8, 145-151.</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* '''Hierarchical restricted Boltzmann machines.''' </div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* '''Hierarchical restricted Boltzmann machines.''' <ins style="font-weight: bold; text-decoration: none;">[http://redwood.berkeley.edu/~amir/vs298/hinton06.pdf Geoff Hinton] has recently developed a hierarchical auto-encoder network, based on the restricted Boltzmann machine, for modelling the structure in data. Implement this network and train on the handwritten digits data or some other dataset of your choosing.</ins></div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* '''Integrate-and-fire model neurons.'''</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* '''Integrate-and-fire model neurons.''' <ins style="font-weight: bold; text-decoration: none;"> </ins></div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Oscillations.''' Oscillations in neural activity are pervasive throughout the brain. What kinds of neural circuits are capable of eliciting oscillating behavior in spiking neurons? How could it be coordinated across large regions of cortex? (ask Fritz Sommer) What role might it play in the processing of information? [http://redwood.berkeley.edu/~amir/vs298/hopfield95.pdf John Hopfield] has suggested that spike timing relative to the phase of an ongoing oscillation could code information. What factors would need to be considered in order to make this idea viable?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Oscillations.''' Oscillations in neural activity are pervasive throughout the brain. What kinds of neural circuits are capable of eliciting oscillating behavior in spiking neurons? How could it be coordinated across large regions of cortex? (ask Fritz Sommer) What role might it play in the processing of information? [http://redwood.berkeley.edu/~amir/vs298/hopfield95.pdf John Hopfield] has suggested that spike timing relative to the phase of an ongoing oscillation could code information. What factors would need to be considered in order to make this idea viable?</div></td></tr>
</table>Brunohttps://rctn.org/w/index.php?title=VS298_(Fall_06):_Suggested_projects&diff=2621&oldid=prevBruno at 04:38, 23 October 20062006-10-23T04:38:39Z<p></p>
<table style="background-color: #fff; color: #202122;" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 04:30, 23 October 2006</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l14">Line 14:</td>
<td colspan="2" class="diff-lineno">Line 14:</td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Sparse codes and associative memory.''' The advantage of storing and recalling patterns using an associative memory as opposed to a conventional computer memory is 1) parallel search, and 2) denoising (recall of an uncorrupted pattern from partial or degraded input). However, associative memory models do not work well with natural data such as images or sound directly. Rather, they are best suited (have highest capacity) for sparse patterns (i.e., patterns with many zeros). Recent work (to be discussed in class) has shown how it is possible to convert natural images and sounds into a sparse format, and there is some evidence for this happening in the brain. See if you can link these ideas in order to store natural images or sounds in an associative memory. </div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Sparse codes and associative memory.''' The advantage of storing and recalling patterns using an associative memory as opposed to a conventional computer memory is 1) parallel search, and 2) denoising (recall of an uncorrupted pattern from partial or degraded input). However, associative memory models do not work well with natural data such as images or sound directly. Rather, they are best suited (have highest capacity) for sparse patterns (i.e., patterns with many zeros). Recent work (to be discussed in class) has shown how it is possible to convert natural images and sounds into a sparse format, and there is some evidence for this happening in the brain. See if you can link these ideas in order to store natural images or sounds in an associative memory. </div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">* '''Entropy of natural images.''' Because of the strong statistical dependencies that exist among pixels in natural images, the true entropy of an image patch is far less than N x (entropy/pixel), where N is the number of pixels. But calculating the true entropy is impossible in practice because it requires collecting the full joint pdf over an image patch, and for any reasonably sized patch this is intractable. Thus it is currently unknown. But there are other ways to estimate the entropy of data beyond direct caculation from the pdf. See for example the recent work of [mailto:djf3@cornell.edu David Field] and colleagues, or methods based on the Hausdorff dimension. Implement one of these methods, or see if you can come up with your own method for estimating the true entropy of images.</ins></div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">* '''ICA applied to EEG.''' ICA has been successfully applied to EEG signals, to separate noise artifacts from brain signals, and also to find separate signal sources within the brain. See if you can get some EEG data from one of the labs on campus (e.g., Klein or Knight labs) and use ICA to reveal the hidden structure in this data. To get started, look at Makeig S, Bell AJ, Jung T-P, Sejnowski TJ (1996) ``Independent Component Analysis of Electroencephalographic Data.'' In: Advances in Neural Information Processing Systems 8, 145-151.</ins></div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Hierarchical restricted Boltzmann machines.''' </div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* '''Hierarchical restricted Boltzmann machines.''' </div></td></tr>
</table>Bruno