VS265: Reading Fall2012

27 Aug

• Dreyfus, H.L. and Dreyfus, S.E. Making a Mind vs. Modeling the Brain: Artificial Intelligence Back at a Branchpoint. Daedalus, Winter 1988.
• Bell, A.J. Levels and loops: the future of artificial intelligence and neuroscience. Phil Trans: Bio Sci. 354:2013--2020 (1999) here or here
• 1973 Lighthill debate on future of AI

29 Aug

• Mead, C. Chapter 1: Introduction and Chapter 4: Neurons from Analog VLSI and Neural Systems, Addison-Wesley, 1989.
• Linear neuron models
• Linear time-invariant systems and convolution
• Simulating differential equations
• Dynamics
• Carandini M, Heeger D (1994) Summation and division by neurons in primate visual cortex. Science, 264: 1333-1336.

Optional:

• Land, MF and Fernald, RD. The Evolution of Eyes, Ann Revs Neuro, 1992.
• Zhang K, Sejnowski TJ (2000) A universal scaling law between gray matter and white matter of cerebral cortex. PNAS, 97: 5621–5626.

05 Sep

• Jordan, M.I. An Introduction to Linear Algebra in Parallel Distributed Processing in McClelland and Rumelhart, Parallel Distributed Processing, MIT Press, 1985.
• Linear neuron models
• Linear algebra primer
• Handout on supervised learning in single-stage feedforward networks

17 Sep

• Handout on supervised learning in multi-layer feedforward networks - "backpropagation"
• Y. LeCun, L. Bottou, G. Orr, and K. Muller (1998) "Efficient BackProp," in Neural Networks: Tricks of the trade, (G. Orr and Muller K., eds.).
• NetTalk demo

24 Sep

• Handout: Hebbian learning and PCA
• HKP Chapters 8 and 9
• PDP Chapter 9 (full text of Michael Jordan's tutorial on linear algebra, including section on eigenvectors)

Optional:

• Atick, Redlich. What does the retina know about natural scenes?, Neural Computation, 1992.
• Dan, Atick, Reid. Efficient Coding of Natural Scenes in the Lateral Geniculate Nucleus: Experimental Test of a Computational Theory, J Neuroscience, 1996.

8 Oct

• Barlow, HB. Single units and sensation: A neuron doctrine for perceptual psychology? Perception, volume 1, pp. 371 -394 (1972)
• Foldiak, P. Forming sparse representations by local anti-Hebbian learning. Biol. Cybern. 64, 165-170 (1990).
• Olshausen BA, Field DJ. Emergence of simple-cell receptive field properties by learning a sparse code for natural images, Nature, 381: 607-609. (1996)

Optional readings:

• Rozell, Johnson, Baraniuk, Olshausen. Sparse Coding via Thresholding and Local Competition in Neural Circuits, Neural Computation 20, 2526–2563 (2008).
• Simoncelli, Olshausen. Natural Image Statistics and Neural Representation, Annu. Rev. Neurosci. 2001. 24:1193–216.
• van Hateren & Ruderman Independent component analysis of natural image sequences, Proc. R. Soc. Lond. B (1998) 265. (blocked sparse coding/ICA of video)
• Olshausen BA Sparse coding of time-varying natural images, ICIP 2003. (convolution sparse coding of video)
• Lewicki MS Efficient coding of natural sounds, Nature Neuroscience, 5 (4): 356-363, 2002. (blocked sparse coding/ICA of sound)
• Smith E, Lewicki MS. Efficient auditory coding, Nature Vol 439 (2006). (convolution sparse coding of sound)

15 Oct

• Ocular dominance column development: Analysis and simulation by Miller, Keller and Stryker.
• A dimension reduction framework for understanding cortical maps by R. Durbin and G. Mitchison.
• The cortical column: a structure without a function by Jonathan C. Horton and Daniel L. Adams

Here are some additional links to papers mentioned in lecture. Optional reading:

- Gary Blasdel, Orientation selectivity, preference, and continuity in monkey striate cortex., J Neurosci, 1992. Another source of many of nice images are in the galleries on Amiram Grinvald's site: [1]

- From Clay Reid's lab, Functional imaging with cellular resolution reveals precise micro-architecture in visual cortex. Make sure you look at the supplementary material and videos on their web site (seems partly broken) [2].

22 Oct

• A Global Geometric Framework for Nonlinear Dimensionality Reduction , Tenenbaum et al., Science 2000.
• Nonlinear Dimensionality Reduction by Locally Linear Embedding, Roweis and Saul, Science 2000.
• On the Local Behavior of Spaces of Natural Images, Carlsson et al., Int J Comput Vis (2008) 76: 1–12.

Additional reading:

• Adaptation to natural facial categories, Michael A. Webster, Daniel Kaping, Yoko Mizokami & Paul Duhamel, Nature, 2004.
• Prototype-referenced shape encoding revealed by high-level aftereffects, David A. Leopold, Alice J. O’Toole, Thomas Vetter and Volker Blanz, Nature, 2001.
• A Morphable Model For The Synthesis Of 3D Faces, Blanz & Vetter 1999.
• Matthew B. Thompson's web page on flashed face distortion effect

24 Oct

• Handout on attractor neural networks
• original Hopfield (1982) paper
• Hopfield (1984) paper
• Marr-Poggio stereo algorithm paper
• Kechen Zhang paper on bump circuits
• Olshausen, Anderson & Van Essen, dynamic routing circuit model
• HKP Chapters 2 and 3

29 Oct

Chris Hillar guest lecture:

• Efficient and Optimal Binary Hopfield Associative Memory Storage Using Minimum Probability Flow
• Robust exponential binary pattern storage in Little-Hopfield networks
• NP-Hard Discrete Quadratic Optimization going into image segmentation (Shi, Malik 2000)

5 Nov

• A probability primer
• Bayesian probability theory and generative models
• Mixture of Gaussians model
• T.J. Loredo, From Laplace to supernova SN1987A: Bayesian inference in astrophysics

19 Nov

• HKP Chapter 7, section 7.1 (Boltzmann machines)

Application to neural data analysis:

• E. Schneidman, M.J. Berry, R. Segev and W. Bialek,Weak pairwise correlations imply strongly correlated network states in a neural population, Nature 4400 (7087) (2006), pp. 1007-1012.
• J. Shlens, G.D. Field, J.L. Gauthier, M.I. Grivich, D. Petrusca, A. Sher, A.M. Litke and E.J. Chichilnisky, The structure of multi-neuron firing patterns in primate retina, J Neurosci 260 (32) (2006), pp. 8254-8266.

21 Nov

• Information theory primer
• Sparse coding and ICA handout
• Jascha Sohl-Dickstein, Natural gradients made quick and dirty
• Jascha Sohl-Dickstein, Natural gradient cookbook
• Bell & Sejnowski, An Information-Maximization Approach to Blind Separation and Blind Deconvolution, Neural Comp, 1995.
• Karklin & Simoncelli, [Efficient coding of natural images with a population of noisy Linear-Nonlinear neurons, NIPS 2011.
• Hyvarinen, Hoyer, Inki, Topographic Independent Component Analysis, Neural Comp, 2001.
• Karklin & Lewicki paper on Learning Higher-Order Structure in Natural Images, Network 2003.
• Shao & Cottrell paper on Recursive ICA, NIPS 2006.

26 Nov

• Robbie Jacobs' notes on Kalman filter
• kalman.m demo script
• Greg Welch's tutorial on Kalman filter
• Dynamic texture models
• Kevin Murphy's HMM tutorial

28 Nov

• Chris Eliasmith, Charlie Anderson, Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems, MIT Press, 2004.

Chapter 4 will be emailed to the class.

• Softky and Koch, The Highly Irregular Firing of Cortical Cells Is Inconsistent with Temporal Integration of Random EPSPs, J Neuroscience, January 1993, 13(1):334-350.
• Mainen and Sejnowski, Reliability of Spike Timing in Neocortical Neurons, Science, Vol 268, 6 June 1995.
• Shadlen and Newsome, Noise, neural codes and cortical organization, Curr Opin in Neur, 1994, 4:569-579.
• Shadlen and Newsom, Is there a signal in the noise?, Current Opin in Neur, 1995, 5:248-250.
• Softky, Simple codes versus efficient codes, Current Opin in Neuro, 1995, 5:239-247.
• Izhikevich, Simple model of spiking neurons, IEEE Trans Neur Networks, 14(6):2003.
• Izhikevich, Which Model to Use for Cortical Spiking Neurons?, IEEE Trans Neur Networks, 15(5):2004.

3 Dec

David Zipser guest lecture:

• HKP section 7.3
• BPTT manual

5 Dec

Pentti Kanerva guest lecture:

• Kanerva, Hyperdimensional Computing