# Difference between revisions of "VS298: Reading"

(→21 Oct) |
|||

(One intermediate revision by the same user not shown) | |||

Line 4: | Line 4: | ||

* Bell, A.J. ''Levels and loops: the future of artificial intelligence and neuroscience''. Phil Trans: Bio Sci. '''354''':2013--2020 (1999) [http://dx.doi.org/10.1098/rstb.1999.0540 here] or [http://www.cnl.salk.edu/~tony/ptrsl.pdf here] | * Bell, A.J. ''Levels and loops: the future of artificial intelligence and neuroscience''. Phil Trans: Bio Sci. '''354''':2013--2020 (1999) [http://dx.doi.org/10.1098/rstb.1999.0540 here] or [http://www.cnl.salk.edu/~tony/ptrsl.pdf here] | ||

− | * Dreyfus, H.L. and Dreyfus, S.E. [http:// | + | * Dreyfus, H.L. and Dreyfus, S.E. [http://redwood.berkeley.edu/amir/vs298/DreyfusDreyfus.pdf ''Making a Mind vs. Modeling the Brain: Artificial Intelligence Back at a Branchpoint'']. Daedalus, Winter 1988. |

− | * Mead, C. [http:// | + | * Mead, C. [http://redwood.berkeley.edu/amir/vs298/Mead.pdf Chapter 1: Introduction] and [http://redwood.berkeley.edu/amir/vs298/Neurons.pdf Chapter 4: Neurons] from ''Analog VLSI and Neural Systems'', Addison-Wesley, 1989. |

− | * Jordan, M.I. [http:// | + | * Jordan, M.I. [http://redwood.berkeley.edu/amir/vs298/PDP.pdf An Introduction to Linear Algebra in Parallel Distributed Processing] in McClelland and Rumelhart, ''Parallel Distributed Processing'', MIT Press, 1985. |

− | * Zhang K, Sejnowski TJ (2000) [http:// | + | * Zhang K, Sejnowski TJ (2000) [http://redwood.berkeley.edu/amir/vs298/zhang-sejnowski.pdf A universal scaling law between gray matter and white matter of cerebral cortex.] PNAS, 97: 5621–5626. |

==== 04 Sep ==== | ==== 04 Sep ==== | ||

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/linear-neuron/linear-neuron-models.html Linear neuron models] |

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/lti-conv/lti-convolution.html Linear time-invariant systems and convolution] |

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/diffeq-sim/diffeq-sim.html Simulating differential equations] |

− | * Carandini M, Heeger D (1994) [http:// | + | * Carandini M, Heeger D (1994) [http://redwood.berkeley.edu/amir/vs298/carandini-heeger.pdf Summation and division by neurons in primate visual cortex.] Science, 264: 1333-1336. |

Optional reading for more background: | Optional reading for more background: | ||

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/linear-algebra/linear-algebra.html Linear algebra primer] |

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/dynamics/dynamics.html Dynamics] |

==== 16 Sep ==== | ==== 16 Sep ==== | ||

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/superlearn1.pdf Handout] on supervised learning in single-stage feedforward networks |

==== 18 Sep ==== | ==== 18 Sep ==== | ||

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/superlearn2.pdf Handout] on supervised learning in multi-layer feedforward networks - "backpropagation" |

− | * Y. LeCun, L. Bottou, G. Orr, and K. Muller (1998) [http:// | + | * Y. LeCun, L. Bottou, G. Orr, and K. Muller (1998) [http://redwood.berkeley.edu/amir/vs298/lecun-98b.pdf "Efficient BackProp,"] in Neural Networks: Tricks of the trade, (G. Orr and Muller K., eds.). |

* [http://www.cnl.salk.edu/ParallelNetsPronounce/index.php NetTalk demo] | * [http://www.cnl.salk.edu/ParallelNetsPronounce/index.php NetTalk demo] | ||

==== 23 Sep ==== | ==== 23 Sep ==== | ||

− | * Handout: [http:// | + | * Handout: [http://redwood.berkeley.edu/amir/vs298/hebb-pca.pdf Hebbian learning and PCA] |

* '''HKP''' Chapter 8 | * '''HKP''' Chapter 8 | ||

− | * '''PDP''' [http:// | + | * '''PDP''' [http://redwood.berkeley.edu/amir/vs298/chap9.pdf Chapter 9] (full text of Michael Jordan's tutorial on linear algebra, including section on eigenvectors) |

==== 25 Sep ==== | ==== 25 Sep ==== | ||

Line 35: | Line 35: | ||

==== 30 Sep ==== | ==== 30 Sep ==== | ||

− | * Foldiak, P. [http:// | + | * Foldiak, P. [http://redwood.berkeley.edu/amir/vs298/foldiak90.pdf Forming sparse representations by local anti-Hebbian learning]. Biol. Cybern. 64, 165-170 (1990). |

− | * Olshausen BA, Field DJ. [http:// | + | * Olshausen BA, Field DJ. [http://redwood.berkeley.edu/amir/vs298/bruno-nature.pdf Emergence of simple-cell receptive field properties by learning a sparse code for natural images], Nature, 381: 607-609. (1996) |

==== 2 Oct ==== | ==== 2 Oct ==== | ||

Optional readings that covers material in lecture in greater depth: | Optional readings that covers material in lecture in greater depth: | ||

− | * Rozell, Johnson, Baraniuk, Olshausen. [http:// | + | * Rozell, Johnson, Baraniuk, Olshausen. [http://redwood.berkeley.edu/amir/vs298/rozell-sparse-coding-nc08.pdf Sparse Coding via Thresholding and Local Competition in Neural Circuits], Neural Computation 20, 2526–2563 (2008). |

− | * Simoncelli, Olshausen. [http:// | + | * Simoncelli, Olshausen. [http://redwood.berkeley.edu/amir/vs298/simoncelli01-reprint.pdf Natural Image Statistics and Neural Representation], Annu. Rev. Neurosci. 2001. 24:1193–216. |

− | * Smith, Lewicki. [http:// | + | * Smith, Lewicki. [http://redwood.berkeley.edu/amir/vs298/smith-lewicki-nature06.pdf Efficient auditory coding], Nature Vol 439 (2006). |

==== 7 Oct ==== | ==== 7 Oct ==== | ||

A handout on sparse coding and on 'ICA', something we haven't yet discussed: | A handout on sparse coding and on 'ICA', something we haven't yet discussed: | ||

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/sparse-coding-handout.pdf Sparse coding and 'ICA' ] |

Dayan and Abbott has a nice section on sparse coding in Chapter 10. This is on the syllabus for unsupervised learning already, but you may want to focus on section 10.3 and 10.4. | Dayan and Abbott has a nice section on sparse coding in Chapter 10. This is on the syllabus for unsupervised learning already, but you may want to focus on section 10.3 and 10.4. | ||

Line 58: | Line 58: | ||

==== 9 Oct ==== | ==== 9 Oct ==== | ||

− | Here are a list of references for David Zipser's talk: [http:// | + | Here are a list of references for David Zipser's talk: [http://redwood.berkeley.edu/amir/vs298/backpropneuralref.pdf pdf]. David also suggested the following chapter in an upcoming book by Thomas J. Anastasio: [http://redwood.berkeley.edu/amir/vs298/zipserchap10.pdf pdf (waiting for approval to post)] |

==== 14 Oct ==== | ==== 14 Oct ==== | ||

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/miller89.pdf Ocular dominance column development: Analysis and simulation] by Miller, Keller and Stryker. |

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/durbin-mitchison.pdf A dimension reduction framework for understanding cortical maps] by R. Durbin and G. Mitchison. |

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/horton05.pdf The cortical column: a structure without a function] by Jonathan C. Horton and Daniel L. Adams |

Here are some additional links to papers mentioned in lecture. Optional reading: | Here are some additional links to papers mentioned in lecture. Optional reading: | ||

− | - Gary Blasdel, [http:// | + | - Gary Blasdel, [http://redwood.berkeley.edu/amir/vs298/blasdel1992.pdf Differential Imaging of Ocular Dominance and Orientation Selectivity in Monkey Striate Cortex], J Neurosci, 1992. Another source of many of nice images are in the galleries on Amiram Grinvald's site: [http://www.weizmann.ac.il/brain/grinvald/] |

- From Clay Reid's lab, [http://www.nature.com/nature/journal/v433/n7026/abs/nature03274.html Functional imaging with cellular resolution reveals precise micro-architecture in visual cortex]. Make sure you look at the supplementary material and videos on their web site (seems partly broken) [http://reid.med.harvard.edu/movies.html]. | - From Clay Reid's lab, [http://www.nature.com/nature/journal/v433/n7026/abs/nature03274.html Functional imaging with cellular resolution reveals precise micro-architecture in visual cortex]. Make sure you look at the supplementary material and videos on their web site (seems partly broken) [http://reid.med.harvard.edu/movies.html]. | ||

Line 74: | Line 74: | ||

==== 16 Oct ==== | ==== 16 Oct ==== | ||

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/tenenbaum-manifold.pdf A Global Geometric Framework for Nonlinear Dimensionality Reduction ], Tenenbaum et al., Science 2000. |

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/roweis-saul-manifold.pdf Nonlinear Dimensionality Reduction by Locally Linear Embedding], Roweis and Saul, Science 2000. |

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/carlsson-ijcv08.pdf On the Local Behavior of Spaces of Natural Images], Carlsson et al., Int J Comput Vis (2008) 76: 1–12. |

Additional reading: | Additional reading: | ||

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/webster-face-adaptation.pdf Adaptation to natural facial categories], Michael A. Webster, Daniel Kaping, Yoko Mizokami & Paul Duhamel, Nature, 2004. |

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/leopold.pdf Prototype-referenced shape encoding revealed by high-level aftereffects], David A. Leopold, Alice J. O’Toole, Thomas Vetter and Volker Blanz, Nature, 2001. |

==== 21 Oct ==== | ==== 21 Oct ==== | ||

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/attractor-networks.pdf Handout] on attractor neural networks |

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/hopfield82.pdf original Hopfield (1982) paper] |

* HKP Chapters 2 and 3 | * HKP Chapters 2 and 3 | ||

==== 23 Oct ==== | ==== 23 Oct ==== | ||

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/hopfield84.pdf Hopfield (1984) paper] |

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/zhang96.pdf Kechen Zhang paper on bump circuits] |

− | * [http:// | + | * [http://redwood.berkeley.edu/amir/vs298/olshausen-etal93.pdf Olshausen, Anderson & Van Essen, dynamic routing circuit model] |

+ | |||

+ | ==== 30 Oct ==== | ||

+ | |||

+ | * [http://redwood.berkeley.edu/amir/vs298/probability.pdf A probability primer] | ||

+ | * [http://redwood.berkeley.edu/amir/vs298/bayes-prob.pdf Bayesian probability theory and generative models] | ||

+ | |||

+ | ==== 4 Nov ==== | ||

+ | |||

+ | * [http://redwood.berkeley.edu/amir/vs298/mog.pdf Mixture of Gaussians model ] |

## Revision as of 06:30, 4 November 2008

For each lecture, we also have a list of optional reading corresponding to ideas discussed in lecture. You may read these if you are interested in the particular topic: Optional Reading

#### 2 Sep

- Bell, A.J.
*Levels and loops: the future of artificial intelligence and neuroscience*. Phil Trans: Bio Sci.**354**:2013--2020 (1999) here or here - Dreyfus, H.L. and Dreyfus, S.E.
*Making a Mind vs. Modeling the Brain: Artificial Intelligence Back at a Branchpoint*. Daedalus, Winter 1988. - Mead, C. Chapter 1: Introduction and Chapter 4: Neurons from
*Analog VLSI and Neural Systems*, Addison-Wesley, 1989. - Jordan, M.I. An Introduction to Linear Algebra in Parallel Distributed Processing in McClelland and Rumelhart,
*Parallel Distributed Processing*, MIT Press, 1985. - Zhang K, Sejnowski TJ (2000) A universal scaling law between gray matter and white matter of cerebral cortex. PNAS, 97: 5621–5626.

#### 04 Sep

- Linear neuron models
- Linear time-invariant systems and convolution
- Simulating differential equations
- Carandini M, Heeger D (1994) Summation and division by neurons in primate visual cortex. Science, 264: 1333-1336.

Optional reading for more background:

#### 16 Sep

- Handout on supervised learning in single-stage feedforward networks

#### 18 Sep

- Handout on supervised learning in multi-layer feedforward networks - "backpropagation"
- Y. LeCun, L. Bottou, G. Orr, and K. Muller (1998) "Efficient BackProp," in Neural Networks: Tricks of the trade, (G. Orr and Muller K., eds.).
- NetTalk demo

#### 23 Sep

- Handout: Hebbian learning and PCA
**HKP**Chapter 8**PDP**Chapter 9 (full text of Michael Jordan's tutorial on linear algebra, including section on eigenvectors)

#### 25 Sep

**HKP**Chapter 9

#### 30 Sep

- Foldiak, P. Forming sparse representations by local anti-Hebbian learning. Biol. Cybern. 64, 165-170 (1990).
- Olshausen BA, Field DJ. Emergence of simple-cell receptive field properties by learning a sparse code for natural images, Nature, 381: 607-609. (1996)

#### 2 Oct

Optional readings that covers material in lecture in greater depth:

- Rozell, Johnson, Baraniuk, Olshausen. Sparse Coding via Thresholding and Local Competition in Neural Circuits, Neural Computation 20, 2526–2563 (2008).

- Simoncelli, Olshausen. Natural Image Statistics and Neural Representation, Annu. Rev. Neurosci. 2001. 24:1193–216.

- Smith, Lewicki. Efficient auditory coding, Nature Vol 439 (2006).

#### 7 Oct

A handout on sparse coding and on 'ICA', something we haven't yet discussed:

Dayan and Abbott has a nice section on sparse coding in Chapter 10. This is on the syllabus for unsupervised learning already, but you may want to focus on section 10.3 and 10.4.

Here is a link to Compressive Sensive Resources at Rice. It has an enormous number of recent papers related to compressed sensing and sparse coding.

#### 9 Oct

Here are a list of references for David Zipser's talk: pdf. David also suggested the following chapter in an upcoming book by Thomas J. Anastasio: pdf (waiting for approval to post)

#### 14 Oct

- Ocular dominance column development: Analysis and simulation by Miller, Keller and Stryker.
- A dimension reduction framework for understanding cortical maps by R. Durbin and G. Mitchison.
- The cortical column: a structure without a function by Jonathan C. Horton and Daniel L. Adams

Here are some additional links to papers mentioned in lecture. Optional reading:

- Gary Blasdel, Differential Imaging of Ocular Dominance and Orientation Selectivity in Monkey Striate Cortex, J Neurosci, 1992. Another source of many of nice images are in the galleries on Amiram Grinvald's site: [1]

- From Clay Reid's lab, Functional imaging with cellular resolution reveals precise micro-architecture in visual cortex. Make sure you look at the supplementary material and videos on their web site (seems partly broken) [2].

#### 16 Oct

- A Global Geometric Framework for Nonlinear Dimensionality Reduction , Tenenbaum et al., Science 2000.

- Nonlinear Dimensionality Reduction by Locally Linear Embedding, Roweis and Saul, Science 2000.

- On the Local Behavior of Spaces of Natural Images, Carlsson et al., Int J Comput Vis (2008) 76: 1–12.

Additional reading:

- Adaptation to natural facial categories, Michael A. Webster, Daniel Kaping, Yoko Mizokami & Paul Duhamel, Nature, 2004.

- Prototype-referenced shape encoding revealed by high-level aftereffects, David A. Leopold, Alice J. O’Toole, Thomas Vetter and Volker Blanz, Nature, 2001.

#### 21 Oct

- Handout on attractor neural networks
- original Hopfield (1982) paper
- HKP Chapters 2 and 3

#### 23 Oct

- Hopfield (1984) paper
- Kechen Zhang paper on bump circuits
- Olshausen, Anderson & Van Essen, dynamic routing circuit model