# Difference between revisions of "VS298: Reading"

(31 intermediate revisions by the same user not shown) | |||

Line 1: | Line 1: | ||

− | |||

− | |||

==== 2 Sep ==== | ==== 2 Sep ==== | ||

Line 8: | Line 6: | ||

* Jordan, M.I. [http://redwood.berkeley.edu/amir/vs298/PDP.pdf An Introduction to Linear Algebra in Parallel Distributed Processing] in McClelland and Rumelhart, ''Parallel Distributed Processing'', MIT Press, 1985. | * Jordan, M.I. [http://redwood.berkeley.edu/amir/vs298/PDP.pdf An Introduction to Linear Algebra in Parallel Distributed Processing] in McClelland and Rumelhart, ''Parallel Distributed Processing'', MIT Press, 1985. | ||

* Zhang K, Sejnowski TJ (2000) [http://redwood.berkeley.edu/amir/vs298/zhang-sejnowski.pdf A universal scaling law between gray matter and white matter of cerebral cortex.] PNAS, 97: 5621–5626. | * Zhang K, Sejnowski TJ (2000) [http://redwood.berkeley.edu/amir/vs298/zhang-sejnowski.pdf A universal scaling law between gray matter and white matter of cerebral cortex.] PNAS, 97: 5621–5626. | ||

+ | |||

+ | Optional: | ||

+ | |||

+ | * Land, MF and Fernald, RD. [http://connes.berkeley.edu/~amir/vs298/landfernald92.pdf The Evolution of Eyes], Ann Revs Neuro, 1992. | ||

+ | |||

+ | * Douglas, R and Martin, K. [http://connes.berkeley.edu/~amir/vs298/douglasmartin2007.pdf Recurrent neuronal circuits in the neocortex], Current Biology, 2007. | ||

==== 04 Sep ==== | ==== 04 Sep ==== | ||

Line 33: | Line 37: | ||

==== 25 Sep ==== | ==== 25 Sep ==== | ||

* '''HKP''' Chapter 9 | * '''HKP''' Chapter 9 | ||

+ | |||

+ | Optional: | ||

+ | |||

+ | * Atick, Redlich. [http://connes.berkeley.edu/~amir/vs298/Atick-Redlich-NC92.pdf What does the retina know about natural scenes?], Neural Computation, 1992. | ||

+ | |||

+ | * Dan, Atick, Reid. [http://www.jneurosci.org/cgi/reprint/16/10/3351.pdf Efficient Coding of Natural Scenes in the Lateral Geniculate Nucleus: Experimental Test of a Computational Theory], J Neuroscience, 1996. | ||

==== 30 Sep ==== | ==== 30 Sep ==== | ||

Line 48: | Line 58: | ||

==== 7 Oct ==== | ==== 7 Oct ==== | ||

− | + | <!--A handout on sparse coding and on 'ICA', something we haven't yet discussed: | |

− | A handout on sparse coding and on 'ICA', something we haven't yet discussed: | + | * [http://redwood.berkeley.edu/amir/vs298/sparse-coding-handout.pdf Sparse coding and 'ICA' ]--> |

− | * [http://redwood.berkeley.edu/amir/vs298/sparse-coding-handout.pdf Sparse coding and 'ICA' ] | ||

Dayan and Abbott has a nice section on sparse coding in Chapter 10. This is on the syllabus for unsupervised learning already, but you may want to focus on section 10.3 and 10.4. | Dayan and Abbott has a nice section on sparse coding in Chapter 10. This is on the syllabus for unsupervised learning already, but you may want to focus on section 10.3 and 10.4. | ||

Line 106: | Line 115: | ||

* [http://redwood.berkeley.edu/amir/vs298/mog.pdf Mixture of Gaussians model ] | * [http://redwood.berkeley.edu/amir/vs298/mog.pdf Mixture of Gaussians model ] | ||

+ | * HKP Chapter 7, section 7.1 | ||

+ | |||

+ | ==== 6 Nov ==== | ||

+ | |||

+ | Some suggested readings for Jon Shlens' talk. | ||

+ | |||

+ | ===== Reviews===== | ||

+ | * S.H. Nirenberg and J.D. Victor, [http://dx.doi.org/10.1016/j.conb.2007.07.002 Analyzing the activity of large populations of neurons: how tractable is the problem?], Curr Opin Neurobiol 17 (4) (2007), pp. 397--400. | ||

+ | |||

+ | * Shlens J, Rieke F, Chichilnisky E. [http://dx.doi.org/10.1016/j.conb.2008.09.010 Synchronized firing in the retina]. Curr Opin Neurobiol. 2008 Oct 27. | ||

+ | |||

+ | =====Theory===== | ||

+ | * S. Amari (2001) [http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=930911&isnumber=20133 Information geometry on hierarchy of probability distributions]. IEEE Trans Inform Theory 47:1701-1711 | ||

+ | |||

+ | * E. Schneidman, S. Still, M.J. Berry and W. Bialek, [http://prola.aps.org/pdf/PRL/v91/i23/e238701 Network information and connected correlations], Phys Rev Lett 91 (2003) 238701. | ||

+ | |||

+ | =====Experiments===== | ||

+ | * E. Schneidman, M.J. Berry, R. Segev and W. Bialek,[http://www.nature.com/nature/journal/v440/n7087/full/nature04701.html Weak pairwise correlations imply strongly correlated network states in a neural population], Nature 4400 (7087) (2006), pp. 1007-1012. | ||

+ | |||

+ | * J. Shlens, G.D. Field, J.L. Gauthier, M.I. Grivich, D. Petrusca, A. Sher, A.M. Litke and E.J. Chichilnisky, [http://www.jneurosci.org/cgi/content/abstract/26/32/8254 The structure of multi-neuron firing patterns in primate retina], J Neurosci 260 (32) (2006), pp. 8254-8266. | ||

+ | |||

+ | * Tang A, Jackson D, Hobbs J, Chen W, Smith JL, Patel H, Prieto A, Petrusca D, Grivich MI, Sher A, Hottowy P, Dabrowski W, Litke AM, Beggs JM. [http://www.jneurosci.org/cgi/content/abstract/28/2/505 A maximum entropy model applied to spatial and temporal correlations from cortical networks in vitro]. J Neurosci. 2008 Jan 9;28(2):505-18. | ||

+ | |||

+ | ==== 18 Nov ==== | ||

+ | |||

+ | * [http://redwood.berkeley.edu/amir/vs298/info-theory.pdf Information theory primer] | ||

+ | * [http://redwood.berkeley.edu/amir/vs298/handout-sparse-08.pdf Sparse coding and ICA handout] | ||

+ | * Bell and Sejnowski, [http://redwood.berkeley.edu/amir/vs298/tony-ica.pdf An Information-Maximization Approach to Blind Separation and Blind Deconvolution], Neural Comp, 1995. | ||

+ | * Hyvarinen, Hoyer, Inki, [http://redwood.berkeley.edu/amir/vs298/TICA.pdf Topographic Independent Component Analysis], Neural Comp, 2001. | ||

+ | |||

+ | ==== 20 Nov ==== | ||

+ | |||

+ | * Robbie Jacobs' [http://www.bcs.rochester.edu/people/robbie/jacobslab/cheat_sheet/sensoryIntegration.pdf notes on Kalman filter] | ||

+ | * Greg Welch's [http://www.cs.unc.edu/~welch/kalman/kalmanIntro.html tutorial on Kalman filter] | ||

+ | * [http://vision.ucla.edu/~doretto/research.html Dynamic texture models] | ||

+ | * Kevin Murphy's [https://redwood.berkeley.edu/amir/vs298/murphy-hmm.pdf HMM tutorial] | ||

+ | |||

+ | ==== 25 Nov ==== | ||

+ | |||

+ | * Chris Eliasmith, Charlie Anderson, [http://books.google.com/books?id=J6jz9s4kbfIC Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems], MIT Press, 2004. | ||

+ | |||

+ | Chapter 4 will be emailed to the class. | ||

+ | |||

+ | * Softky and Koch, [http://redwood.berkeley.edu/amir/vs298/softky-koch-jn93.pdf The Highly Irregular Firing of Cortical Cells Is Inconsistent with Temporal Integration of Random EPSPs], J Neuroscience, January 1993, 13(1):334-350. | ||

+ | * Mainen and Sejnowski, [http://redwood.berkeley.edu/amir/vs298/mainen-sejnowski.pdf Reliability of Spike Timing in Neocortical Neurons], Science, Vol 268, 6 June 1995. | ||

+ | * Shadlen and Newsome, [http://redwood.berkeley.edu/amir/vs298/shadlen-newsome1.pdf Noise, neural codes and cortical organization], Curr Opin in Neur, 1994, 4:569-579. | ||

+ | * Shadlen and Newsom, [http://redwood.berkeley.edu/amir/vs298/shadlen-newsome1.pdf Is there a signal in the noise?], Current Opin in Neur, 1995, 5:248-250. | ||

+ | * Softky, [http://redwood.berkeley.edu/amir/vs298/softky-commentary.pdf Simple codes versus efficient codes], Current Opin in Neuro, 1995, 5:239-247. | ||

+ | * Izhikevich, [http://redwood.berkeley.edu/amir/vs298/izhikevich-nn03.pdf Simple model of spiking neurons], IEEE Trans Neur Networks, 14(6):2003. | ||

+ | * Izhikevich, [http://redwood.berkeley.edu/amir/vs298/izhikevich-which-nn04.pdf Which Model to Use for Cortical Spiking Neurons?], IEEE Trans Neur Networks, 15(5):2004. | ||

+ | |||

+ | ==== 4 Dec ==== | ||

+ | |||

+ | * A.J. Bell, [http://redwood.berkeley.edu/amir/vs298/bell-cross-level.pdf Towards a Cross-Level Theory of Neural Learning]. |

## Latest revision as of 07:13, 11 December 2008

#### 2 Sep

- Bell, A.J.
*Levels and loops: the future of artificial intelligence and neuroscience*. Phil Trans: Bio Sci.**354**:2013--2020 (1999) here or here - Dreyfus, H.L. and Dreyfus, S.E.
*Making a Mind vs. Modeling the Brain: Artificial Intelligence Back at a Branchpoint*. Daedalus, Winter 1988. - Mead, C. Chapter 1: Introduction and Chapter 4: Neurons from
*Analog VLSI and Neural Systems*, Addison-Wesley, 1989. - Jordan, M.I. An Introduction to Linear Algebra in Parallel Distributed Processing in McClelland and Rumelhart,
*Parallel Distributed Processing*, MIT Press, 1985. - Zhang K, Sejnowski TJ (2000) A universal scaling law between gray matter and white matter of cerebral cortex. PNAS, 97: 5621–5626.

Optional:

- Land, MF and Fernald, RD. The Evolution of Eyes, Ann Revs Neuro, 1992.

- Douglas, R and Martin, K. Recurrent neuronal circuits in the neocortex, Current Biology, 2007.

#### 04 Sep

- Linear neuron models
- Linear time-invariant systems and convolution
- Simulating differential equations
- Carandini M, Heeger D (1994) Summation and division by neurons in primate visual cortex. Science, 264: 1333-1336.

Optional reading for more background:

#### 16 Sep

- Handout on supervised learning in single-stage feedforward networks

#### 18 Sep

- Handout on supervised learning in multi-layer feedforward networks - "backpropagation"
- Y. LeCun, L. Bottou, G. Orr, and K. Muller (1998) "Efficient BackProp," in Neural Networks: Tricks of the trade, (G. Orr and Muller K., eds.).
- NetTalk demo

#### 23 Sep

- Handout: Hebbian learning and PCA
**HKP**Chapter 8**PDP**Chapter 9 (full text of Michael Jordan's tutorial on linear algebra, including section on eigenvectors)

#### 25 Sep

**HKP**Chapter 9

Optional:

- Atick, Redlich. What does the retina know about natural scenes?, Neural Computation, 1992.

- Dan, Atick, Reid. Efficient Coding of Natural Scenes in the Lateral Geniculate Nucleus: Experimental Test of a Computational Theory, J Neuroscience, 1996.

#### 30 Sep

- Foldiak, P. Forming sparse representations by local anti-Hebbian learning. Biol. Cybern. 64, 165-170 (1990).
- Olshausen BA, Field DJ. Emergence of simple-cell receptive field properties by learning a sparse code for natural images, Nature, 381: 607-609. (1996)

#### 2 Oct

Optional readings that covers material in lecture in greater depth:

- Rozell, Johnson, Baraniuk, Olshausen. Sparse Coding via Thresholding and Local Competition in Neural Circuits, Neural Computation 20, 2526–2563 (2008).

- Simoncelli, Olshausen. Natural Image Statistics and Neural Representation, Annu. Rev. Neurosci. 2001. 24:1193–216.

- Smith, Lewicki. Efficient auditory coding, Nature Vol 439 (2006).

#### 7 Oct

Dayan and Abbott has a nice section on sparse coding in Chapter 10. This is on the syllabus for unsupervised learning already, but you may want to focus on section 10.3 and 10.4.

Here is a link to Compressive Sensive Resources at Rice. It has an enormous number of recent papers related to compressed sensing and sparse coding.

#### 9 Oct

Here are a list of references for David Zipser's talk: pdf. David also suggested the following chapter in an upcoming book by Thomas J. Anastasio: pdf (waiting for approval to post)

#### 14 Oct

- Ocular dominance column development: Analysis and simulation by Miller, Keller and Stryker.
- A dimension reduction framework for understanding cortical maps by R. Durbin and G. Mitchison.
- The cortical column: a structure without a function by Jonathan C. Horton and Daniel L. Adams

Here are some additional links to papers mentioned in lecture. Optional reading:

- Gary Blasdel, Differential Imaging of Ocular Dominance and Orientation Selectivity in Monkey Striate Cortex, J Neurosci, 1992. Another source of many of nice images are in the galleries on Amiram Grinvald's site: [1]

- From Clay Reid's lab, Functional imaging with cellular resolution reveals precise micro-architecture in visual cortex. Make sure you look at the supplementary material and videos on their web site (seems partly broken) [2].

#### 16 Oct

- A Global Geometric Framework for Nonlinear Dimensionality Reduction , Tenenbaum et al., Science 2000.

- Nonlinear Dimensionality Reduction by Locally Linear Embedding, Roweis and Saul, Science 2000.

- On the Local Behavior of Spaces of Natural Images, Carlsson et al., Int J Comput Vis (2008) 76: 1–12.

Additional reading:

- Adaptation to natural facial categories, Michael A. Webster, Daniel Kaping, Yoko Mizokami & Paul Duhamel, Nature, 2004.

- Prototype-referenced shape encoding revealed by high-level aftereffects, David A. Leopold, Alice J. O’Toole, Thomas Vetter and Volker Blanz, Nature, 2001.

#### 21 Oct

- Handout on attractor neural networks
- original Hopfield (1982) paper
- HKP Chapters 2 and 3

#### 23 Oct

- Hopfield (1984) paper
- Kechen Zhang paper on bump circuits
- Olshausen, Anderson & Van Essen, dynamic routing circuit model

#### 30 Oct

#### 4 Nov

- Mixture of Gaussians model
- HKP Chapter 7, section 7.1

#### 6 Nov

Some suggested readings for Jon Shlens' talk.

##### Reviews

- S.H. Nirenberg and J.D. Victor, Analyzing the activity of large populations of neurons: how tractable is the problem?, Curr Opin Neurobiol 17 (4) (2007), pp. 397--400.

- Shlens J, Rieke F, Chichilnisky E. Synchronized firing in the retina. Curr Opin Neurobiol. 2008 Oct 27.

##### Theory

- S. Amari (2001) Information geometry on hierarchy of probability distributions. IEEE Trans Inform Theory 47:1701-1711

- E. Schneidman, S. Still, M.J. Berry and W. Bialek, Network information and connected correlations, Phys Rev Lett 91 (2003) 238701.

##### Experiments

- E. Schneidman, M.J. Berry, R. Segev and W. Bialek,Weak pairwise correlations imply strongly correlated network states in a neural population, Nature 4400 (7087) (2006), pp. 1007-1012.

- J. Shlens, G.D. Field, J.L. Gauthier, M.I. Grivich, D. Petrusca, A. Sher, A.M. Litke and E.J. Chichilnisky, The structure of multi-neuron firing patterns in primate retina, J Neurosci 260 (32) (2006), pp. 8254-8266.

- Tang A, Jackson D, Hobbs J, Chen W, Smith JL, Patel H, Prieto A, Petrusca D, Grivich MI, Sher A, Hottowy P, Dabrowski W, Litke AM, Beggs JM. A maximum entropy model applied to spatial and temporal correlations from cortical networks in vitro. J Neurosci. 2008 Jan 9;28(2):505-18.

#### 18 Nov

- Information theory primer
- Sparse coding and ICA handout
- Bell and Sejnowski, An Information-Maximization Approach to Blind Separation and Blind Deconvolution, Neural Comp, 1995.
- Hyvarinen, Hoyer, Inki, Topographic Independent Component Analysis, Neural Comp, 2001.

#### 20 Nov

- Robbie Jacobs' notes on Kalman filter
- Greg Welch's tutorial on Kalman filter
- Dynamic texture models
- Kevin Murphy's HMM tutorial

#### 25 Nov

- Chris Eliasmith, Charlie Anderson, Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems, MIT Press, 2004.

Chapter 4 will be emailed to the class.

- Softky and Koch, The Highly Irregular Firing of Cortical Cells Is Inconsistent with Temporal Integration of Random EPSPs, J Neuroscience, January 1993, 13(1):334-350.
- Mainen and Sejnowski, Reliability of Spike Timing in Neocortical Neurons, Science, Vol 268, 6 June 1995.
- Shadlen and Newsome, Noise, neural codes and cortical organization, Curr Opin in Neur, 1994, 4:569-579.
- Shadlen and Newsom, Is there a signal in the noise?, Current Opin in Neur, 1995, 5:248-250.
- Softky, Simple codes versus efficient codes, Current Opin in Neuro, 1995, 5:239-247.
- Izhikevich, Simple model of spiking neurons, IEEE Trans Neur Networks, 14(6):2003.
- Izhikevich, Which Model to Use for Cortical Spiking Neurons?, IEEE Trans Neur Networks, 15(5):2004.

#### 4 Dec

- A.J. Bell, Towards a Cross-Level Theory of Neural Learning.