# VS265: Reading Fall2010: Difference between revisions

No edit summary |
(Updated the dates back to what they were for Fall 2010) |
||

(31 intermediate revisions by 2 users not shown) | |||

Line 25: | Line 25: | ||

==== 21 Sep ==== | ==== 21 Sep ==== | ||

* Handout: [http://redwood.berkeley.edu/vs265/hebb-pca.pdf Hebbian learning and PCA] | * Handout: [http://redwood.berkeley.edu/vs265/hebb-pca-handout.pdf Hebbian learning and PCA] | ||

* '''HKP''' Chapters 8 and 9 | * '''HKP''' Chapters 8 and 9 | ||

* '''PDP''' [http://redwood.berkeley.edu/vs265/chap9.pdf Chapter 9] (full text of Michael Jordan's tutorial on linear algebra, including section on eigenvectors) | * '''PDP''' [http://redwood.berkeley.edu/vs265/chap9.pdf Chapter 9] (full text of Michael Jordan's tutorial on linear algebra, including section on eigenvectors) | ||

Line 32: | Line 32: | ||

* Atick, Redlich. [http://redwood.berkeley.edu/vs265/Atick-Redlich-NC92.pdf What does the retina know about natural scenes?], Neural Computation, 1992. | * Atick, Redlich. [http://redwood.berkeley.edu/vs265/Atick-Redlich-NC92.pdf What does the retina know about natural scenes?], Neural Computation, 1992. | ||

* Dan, Atick, Reid. [http://www.jneurosci.org/cgi/reprint/16/10/3351.pdf Efficient Coding of Natural Scenes in the Lateral Geniculate Nucleus: Experimental Test of a Computational Theory], J Neuroscience, 1996. | * Dan, Atick, Reid. [http://www.jneurosci.org/cgi/reprint/16/10/3351.pdf Efficient Coding of Natural Scenes in the Lateral Geniculate Nucleus: Experimental Test of a Computational Theory], J Neuroscience, 1996. | ||

==== 28 Sep ==== | |||

* Foldiak, P. [http://redwood.berkeley.edu/vs265/foldiak90.pdf Forming sparse representations by local anti-Hebbian learning]. Biol. Cybern. 64, 165-170 (1990). | |||

* Olshausen BA, Field DJ. [http://redwood.berkeley.edu/vs265/bruno-nature.pdf Emergence of simple-cell receptive field properties by learning a sparse code for natural images], Nature, 381: 607-609. (1996) | |||

Optional readings: | |||

* Rozell, Johnson, Baraniuk, Olshausen. [http://redwood.berkeley.edu/vs265/rozell-sparse-coding-nc08.pdf Sparse Coding via Thresholding and Local Competition in Neural Circuits], Neural Computation 20, 2526–2563 (2008). | |||

* Simoncelli, Olshausen. [http://redwood.berkeley.edu/vs265/simoncelli01-reprint.pdf Natural Image Statistics and Neural Representation], Annu. Rev. Neurosci. 2001. 24:1193–216. | |||

* Smith, Lewicki. [http://redwood.berkeley.edu/vs265/smith-lewicki-nature06.pdf Efficient auditory coding], Nature Vol 439 (2006). | |||

==== 5 Oct ==== | |||

* [http://redwood.berkeley.edu/vs265/miller89.pdf Ocular dominance column development: Analysis and simulation] by Miller, Keller and Stryker. | |||

* [http://redwood.berkeley.edu/vs265/durbin-mitchison.pdf A dimension reduction framework for understanding cortical maps] by R. Durbin and G. Mitchison. | |||

* [http://redwood.berkeley.edu/vs265/horton05.pdf The cortical column: a structure without a function] by Jonathan C. Horton and Daniel L. Adams | |||

Here are some additional links to papers mentioned in lecture. Optional reading: | |||

- Gary Blasdel, [http://redwood.berkeley.edu/vs265/blasdel1992.pdf Orientation selectivity, preference, and continuity in monkey striate cortex.], J Neurosci, 1992. Another source of many of nice images are in the galleries on Amiram Grinvald's site: [http://www.weizmann.ac.il/brain/grinvald/] | |||

- From Clay Reid's lab, [http://www.nature.com/nature/journal/v433/n7026/abs/nature03274.html Functional imaging with cellular resolution reveals precise micro-architecture in visual cortex]. Make sure you look at the supplementary material and videos on their web site (seems partly broken) [http://reid.med.harvard.edu/movies.html]. | |||

==== 7 Oct ==== | |||

* [http://redwood.berkeley.edu/vs265/tenenbaum-manifold.pdf A Global Geometric Framework for Nonlinear Dimensionality Reduction ], Tenenbaum et al., Science 2000. | |||

* [http://redwood.berkeley.edu/vs265/roweis-saul-manifold.pdf Nonlinear Dimensionality Reduction by Locally Linear Embedding], Roweis and Saul, Science 2000. | |||

* [http://redwood.berkeley.edu/vs265/carlsson-ijcv08.pdf On the Local Behavior of Spaces of Natural Images], Carlsson et al., Int J Comput Vis (2008) 76: 1–12. | |||

Additional reading: | |||

* [http://redwood.berkeley.edu/vs265/webster-face-adaptation.pdf Adaptation to natural facial categories], Michael A. Webster, Daniel Kaping, Yoko Mizokami & Paul Duhamel, Nature, 2004. | |||

* [http://redwood.berkeley.edu/vs265/leopold.pdf Prototype-referenced shape encoding revealed by high-level aftereffects], David A. Leopold, Alice J. O’Toole, Thomas Vetter and Volker Blanz, Nature, 2001. | |||

==== 12/14 Oct ==== | |||

* [http://redwood.berkeley.edu/vs265/attractor-networks.pdf Handout] on attractor neural networks | |||

* [http://redwood.berkeley.edu/vs265/hopfield82.pdf original Hopfield (1982) paper] | |||

* [http://redwood.berkeley.edu/vs265/hopfield84.pdf Hopfield (1984) paper] | |||

* [http://redwood.berkeley.edu/vs265/marr-poggio-science76.pdf Marr-Poggio stereo algorithm paper] | |||

* [http://redwood.berkeley.edu/vs265/zhang96.pdf Kechen Zhang paper on bump circuits] | |||

* [http://redwood.berkeley.edu/vs265/olshausen-etal93.pdf Olshausen, Anderson & Van Essen, dynamic routing circuit model] | |||

* HKP Chapters 2 and 3 | |||

==== 19 Oct (David Zipser guest lecture) ==== | |||

* [http://redwood.berkeley.edu/vs265/zipser-manual.pdf manual] for David Zipser's BPTT simulator | |||

==== 26 Oct ==== | |||

* [http://redwood.berkeley.edu/vs265/probability.pdf A probability primer] | |||

* [http://redwood.berkeley.edu/vs265/bayes-prob.pdf Bayesian probability theory and generative models] | |||

* [http://redwood.berkeley.edu/vs265/mog.pdf Mixture of Gaussians model ] | |||

==== 2/4 Nov ==== | |||

* HKP Chapter 7, section 7.1 | |||

Application to neural data analysis: | |||

* E. Schneidman, M.J. Berry, R. Segev and W. Bialek,[http://www.nature.com/nature/journal/v440/n7087/full/nature04701.html Weak pairwise correlations imply strongly correlated network states in a neural population], Nature 4400 (7087) (2006), pp. 1007-1012. | |||

* J. Shlens, G.D. Field, J.L. Gauthier, M.I. Grivich, D. Petrusca, A. Sher, A.M. Litke and E.J. Chichilnisky, [http://www.jneurosci.org/cgi/content/abstract/26/32/8254 The structure of multi-neuron firing patterns in primate retina], J Neurosci 260 (32) (2006), pp. 8254-8266. | |||

==== 16 Nov ==== | |||

* [http://redwood.berkeley.edu/vs265/info-theory.pdf Information theory primer] | |||

* [http://redwood.berkeley.edu/vs265/handout-sparse-08.pdf Sparse coding and ICA handout] | |||

* Bell and Sejnowski, [http://redwood.berkeley.edu/vs265/tony-ica.pdf An Information-Maximization Approach to Blind Separation and Blind Deconvolution], Neural Comp, 1995. | |||

* Hyvarinen, Hoyer, Inki, [http://redwood.berkeley.edu/vs265/TICA.pdf Topographic Independent Component Analysis], Neural Comp, 2001. | |||

* Karklin & Lewicki paper on [http://redwood.berkeley.edu/vs265/karklin-lewicki2003.pdf Learning Higher-Order Structure in Natural Images], Network 2003. | |||

* Shao & Cottrell paper on [http://redwood.berkeley.edu/vs265/hshan-nips06.pdf Recursive ICA], NIPS 2006. | |||

==== 18 Nov ==== | |||

* Robbie Jacobs' [http://www.bcs.rochester.edu/people/robbie/jacobslab/cheat_sheet/sensoryIntegration.pdf notes on Kalman filter] | |||

* [http://redwood.berkeley.edu/vs265/kalman.m kalman.m] demo script | |||

* Greg Welch's [http://www.cs.unc.edu/~welch/kalman/kalmanIntro.html tutorial on Kalman filter] | |||

* [http://vision.ucla.edu/~doretto/research.html Dynamic texture models] | |||

* Kevin Murphy's [http://redwood.berkeley.edu/vs265/murphy-hmm.pdf HMM tutorial] | |||

==== 23 Nov ==== | |||

* Chris Eliasmith, Charlie Anderson, [http://books.google.com/books?id=J6jz9s4kbfIC Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems], MIT Press, 2004. | |||

Chapter 4 will be emailed to the class. | |||

* Softky and Koch, [http://redwood.berkeley.edu/vs265/softky-koch-jn93.pdf The Highly Irregular Firing of Cortical Cells Is Inconsistent with Temporal Integration of Random EPSPs], J Neuroscience, January 1993, 13(1):334-350. | |||

* Mainen and Sejnowski, [http://redwood.berkeley.edu/vs265/mainen-sejnowski.pdf Reliability of Spike Timing in Neocortical Neurons], Science, Vol 268, 6 June 1995. | |||

* Shadlen and Newsome, [http://redwood.berkeley.edu/vs265/shadlen-newsome1.pdf Noise, neural codes and cortical organization], Curr Opin in Neur, 1994, 4:569-579. | |||

* Shadlen and Newsom, [http://redwood.berkeley.edu/vs265/shadlen-newsome1.pdf Is there a signal in the noise?], Current Opin in Neur, 1995, 5:248-250. | |||

* Softky, [http://redwood.berkeley.edu/vs265/softky-commentary.pdf Simple codes versus efficient codes], Current Opin in Neuro, 1995, 5:239-247. | |||

* Izhikevich, [http://redwood.berkeley.edu/vs265/izhikevich-nn03.pdf Simple model of spiking neurons], IEEE Trans Neur Networks, 14(6):2003. | |||

* Izhikevich, [http://redwood.berkeley.edu/vs265/izhikevich-which-nn04.pdf Which Model to Use for Cortical Spiking Neurons?], IEEE Trans Neur Networks, 15(5):2004. | |||

==== 2 Dec (Jeff Hawkins guest lecture) ==== | |||

* Numenta document on [http://redwood.berkeley.edu/vs265/HTM_CorticalLearningAlgorithms.pdf Hierarchical Temporal Memory] | |||

==== 7 Dec (Paul Rhodes guest lecture) ==== | |||

* Niell, Meyer, & Smith, [http://redwood.berkeley.edu/vs265/niell-smith-nn04.pdf In vivo imaging of synapse formation on a growing dendritic arbor], Nature Neuroscience, 7, 254-260. | |||

* Meyer, & Smith, [http://redwood.berkeley.edu/vs265/meyer-smith-jn06.pdf Evidence from In Vivo Imaging That Synaptogenesis Guides the Growth and Branching of Axonal Arbors by Two Distinct Mechanisms], Journal of Neuroscience, 26, 3604-3614. | |||

==== 9 Dec (Pentti Kanerva guest lecture) ==== | |||

* Kanerva, P [http://www.amazon.com/Sparse-Distributed-Memory-Bradford-Books/dp/0262111322 Sparse Distributed Memory] |

## Latest revision as of 02:55, 28 August 2012

#### 26 Aug

- Dreyfus, H.L. and Dreyfus, S.E.
*Making a Mind vs. Modeling the Brain: Artificial Intelligence Back at a Branchpoint*. Daedalus, Winter 1988. - Bell, A.J.
*Levels and loops: the future of artificial intelligence and neuroscience*. Phil Trans: Bio Sci.**354**:2013--2020 (1999) here or here

Optional:

- Land, MF and Fernald, RD. The Evolution of Eyes, Ann Revs Neuro, 1992.

#### 31 Aug

- Mead, C. Chapter 1: Introduction and Chapter 4: Neurons from
*Analog VLSI and Neural Systems*, Addison-Wesley, 1989. - Linear time-invariant systems and convolution
- Simulating differential equations
- Dynamics
- Carandini M, Heeger D (1994) Summation and division by neurons in primate visual cortex. Science, 264: 1333-1336.

#### 02 Sep

- Jordan, M.I. An Introduction to Linear Algebra in Parallel Distributed Processing in McClelland and Rumelhart,
*Parallel Distributed Processing*, MIT Press, 1985. - Linear neuron models
- Linear algebra primer

#### 07 Sep

- Handout on supervised learning in single-stage feedforward networks
- Handout on supervised learning in multi-layer feedforward networks - "backpropagation"
- Y. LeCun, L. Bottou, G. Orr, and K. Muller (1998) "Efficient BackProp," in Neural Networks: Tricks of the trade, (G. Orr and Muller K., eds.).
- NetTalk demo

#### 21 Sep

- Handout: Hebbian learning and PCA
**HKP**Chapters 8 and 9**PDP**Chapter 9 (full text of Michael Jordan's tutorial on linear algebra, including section on eigenvectors)

Optional:

- Atick, Redlich. What does the retina know about natural scenes?, Neural Computation, 1992.
- Dan, Atick, Reid. Efficient Coding of Natural Scenes in the Lateral Geniculate Nucleus: Experimental Test of a Computational Theory, J Neuroscience, 1996.

#### 28 Sep

- Foldiak, P. Forming sparse representations by local anti-Hebbian learning. Biol. Cybern. 64, 165-170 (1990).
- Olshausen BA, Field DJ. Emergence of simple-cell receptive field properties by learning a sparse code for natural images, Nature, 381: 607-609. (1996)

Optional readings:

- Rozell, Johnson, Baraniuk, Olshausen. Sparse Coding via Thresholding and Local Competition in Neural Circuits, Neural Computation 20, 2526–2563 (2008).
- Simoncelli, Olshausen. Natural Image Statistics and Neural Representation, Annu. Rev. Neurosci. 2001. 24:1193–216.
- Smith, Lewicki. Efficient auditory coding, Nature Vol 439 (2006).

#### 5 Oct

- Ocular dominance column development: Analysis and simulation by Miller, Keller and Stryker.
- A dimension reduction framework for understanding cortical maps by R. Durbin and G. Mitchison.
- The cortical column: a structure without a function by Jonathan C. Horton and Daniel L. Adams

Here are some additional links to papers mentioned in lecture. Optional reading:

- Gary Blasdel, Orientation selectivity, preference, and continuity in monkey striate cortex., J Neurosci, 1992. Another source of many of nice images are in the galleries on Amiram Grinvald's site: [1]

- From Clay Reid's lab, Functional imaging with cellular resolution reveals precise micro-architecture in visual cortex. Make sure you look at the supplementary material and videos on their web site (seems partly broken) [2].

#### 7 Oct

- A Global Geometric Framework for Nonlinear Dimensionality Reduction , Tenenbaum et al., Science 2000.
- Nonlinear Dimensionality Reduction by Locally Linear Embedding, Roweis and Saul, Science 2000.
- On the Local Behavior of Spaces of Natural Images, Carlsson et al., Int J Comput Vis (2008) 76: 1–12.

Additional reading:

- Adaptation to natural facial categories, Michael A. Webster, Daniel Kaping, Yoko Mizokami & Paul Duhamel, Nature, 2004.
- Prototype-referenced shape encoding revealed by high-level aftereffects, David A. Leopold, Alice J. O’Toole, Thomas Vetter and Volker Blanz, Nature, 2001.

#### 12/14 Oct

- Handout on attractor neural networks
- original Hopfield (1982) paper
- Hopfield (1984) paper
- Marr-Poggio stereo algorithm paper
- Kechen Zhang paper on bump circuits
- Olshausen, Anderson & Van Essen, dynamic routing circuit model
- HKP Chapters 2 and 3

#### 19 Oct (David Zipser guest lecture)

- manual for David Zipser's BPTT simulator

#### 26 Oct

#### 2/4 Nov

- HKP Chapter 7, section 7.1

Application to neural data analysis:

- E. Schneidman, M.J. Berry, R. Segev and W. Bialek,Weak pairwise correlations imply strongly correlated network states in a neural population, Nature 4400 (7087) (2006), pp. 1007-1012.
- J. Shlens, G.D. Field, J.L. Gauthier, M.I. Grivich, D. Petrusca, A. Sher, A.M. Litke and E.J. Chichilnisky, The structure of multi-neuron firing patterns in primate retina, J Neurosci 260 (32) (2006), pp. 8254-8266.

#### 16 Nov

- Information theory primer
- Sparse coding and ICA handout
- Bell and Sejnowski, An Information-Maximization Approach to Blind Separation and Blind Deconvolution, Neural Comp, 1995.
- Hyvarinen, Hoyer, Inki, Topographic Independent Component Analysis, Neural Comp, 2001.
- Karklin & Lewicki paper on Learning Higher-Order Structure in Natural Images, Network 2003.
- Shao & Cottrell paper on Recursive ICA, NIPS 2006.

#### 18 Nov

- Robbie Jacobs' notes on Kalman filter
- kalman.m demo script
- Greg Welch's tutorial on Kalman filter
- Dynamic texture models
- Kevin Murphy's HMM tutorial

#### 23 Nov

- Chris Eliasmith, Charlie Anderson, Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems, MIT Press, 2004.

Chapter 4 will be emailed to the class.

- Softky and Koch, The Highly Irregular Firing of Cortical Cells Is Inconsistent with Temporal Integration of Random EPSPs, J Neuroscience, January 1993, 13(1):334-350.
- Mainen and Sejnowski, Reliability of Spike Timing in Neocortical Neurons, Science, Vol 268, 6 June 1995.
- Shadlen and Newsome, Noise, neural codes and cortical organization, Curr Opin in Neur, 1994, 4:569-579.
- Shadlen and Newsom, Is there a signal in the noise?, Current Opin in Neur, 1995, 5:248-250.
- Softky, Simple codes versus efficient codes, Current Opin in Neuro, 1995, 5:239-247.
- Izhikevich, Simple model of spiking neurons, IEEE Trans Neur Networks, 14(6):2003.
- Izhikevich, Which Model to Use for Cortical Spiking Neurons?, IEEE Trans Neur Networks, 15(5):2004.

#### 2 Dec (Jeff Hawkins guest lecture)

- Numenta document on Hierarchical Temporal Memory

#### 7 Dec (Paul Rhodes guest lecture)

- Niell, Meyer, & Smith, In vivo imaging of synapse formation on a growing dendritic arbor, Nature Neuroscience, 7, 254-260.

- Meyer, & Smith, Evidence from In Vivo Imaging That Synaptogenesis Guides the Growth and Branching of Axonal Arbors by Two Distinct Mechanisms, Journal of Neuroscience, 26, 3604-3614.

#### 9 Dec (Pentti Kanerva guest lecture)

- Kanerva, P Sparse Distributed Memory