VS265: Reading

Aug 28: Introduction

• HKP chapter 1
• Dreyfus, H.L. and Dreyfus, S.E. Making a Mind vs. Modeling the Brain: Artificial Intelligence Back at a Branchpoint. Daedalus, Winter 1988.
• Bell, A.J. Levels and loops: the future of artificial intelligence and neuroscience. Phil Trans: Bio Sci. 354:2013--2020 (1999) here or here
• 1973 Lighthill debate on future of AI

Optional:

• Land, MF and Fernald, RD. The Evolution of Eyes, Ann Revs Neuro, 1992.
• Zhang K, Sejnowski TJ (2000) A universal scaling law between gray matter and white matter of cerebral cortex. PNAS, 97: 5621–5626.
• O'Rourke, N.A et al. "Deep molecular diversity of mammalian synapses: why it matters and how to measure it." Nature Reviews Neurosci. 13, (2012)
• Stephen Smith Array Tomography movies
• Solari & Stoner, Cognitive Consilience

Sept 2: Neuron models

• Mead, C. Chapter 1: Introduction and Chapter 4: Neurons from Analog VLSI and Neural Systems, Addison-Wesley, 1989.
• Carandini M, Heeger D (1994) Summation and division by neurons in primate visual cortex. Science, 264: 1333-1336.

Background reading on dynamics, linear time-invariant systems and convolution, and differential equations:

• Dynamics
• Linear time-invariant systems and convolution
• Simulating differential equations

Sept 4: Linear neuron, Perceptron

• HKP chapter 5, DJCM chapters 38-40, 44, DA chapter 8 (sec. 4-6)
• Linear neuron models

Background on linear algebra:

• Linear algebra primer
• Jordan, M.I. An Introduction to Linear Algebra in Parallel Distributed Processing in McClelland and Rumelhart, Parallel Distributed Processing, MIT Press, 1985.

Sept 11: Multicompartment models, dendritic integration (Rhodes guest lecture)

• Koch, Single Neuron Computation, Chapter 19 pdf
• Rhodes P (1999) Functional Implications of Active Currents in the Dendrites of Pyramidal Neurons
• Schiller J (2003) Submillisecond Precision of the Input–Output Transformation Function Mediated by Fast Sodium Dendritic Spikes in Basal Dendrites of CA1 Pyramidal Neurons

Sept. 16, 18: Supervised learning

• HKP Chapters 5, 6
• Handout on supervised learning in single-stage feedforward networks
• Handout on supervised learning in multi-layer feedforward networks - "back propagation"

Further reading:

• Y. LeCun, L. Bottou, G. Orr, and K. Muller (1998) "Efficient BackProp," in Neural Networks: Tricks of the trade, (G. Orr and Muller K., eds.).
• NetTalk demo

Sept. 23, 24: Unsupervised learning

• HKP Chapters 8 and 9, DJCM chapter 36, DA chapter 8, 10
• Handout: Hebbian learning and PCA
• PDP Chapter 9 (full text of Michael Jordan's tutorial on linear algebra, including section on eigenvectors)

Optional:

• Atick, Redlich. What does the retina know about natural scenes?, Neural Computation, 1992.
• Dan, Atick, Reid. Efficient Coding of Natural Scenes in the Lateral Geniculate Nucleus: Experimental Test of a Computational Theory, J Neuroscience, 1996.

Sept 30, Oct 2: Attractor Networks and Associative Memories (Sommer guest lectures)

• "HKP" Chapter 2 and 3 (sec. 3.3-3.5), 7 (sec. 7.2-7.3), DJCM chapter 42, DA chapter 7
• Handout on attractor networks - their learning, dynamics and how they differ from feed-forward networks
• Hopfield82
• Hopfield84
• Willshaw69

Oct 7: Ecological utility and the mythical neural code (Feldman guest lecture)

• Feldman10 Ecological utility and the mythical neural code

Oct 9: Hyperdimensional computing (Kanerva guest lecture)

• Kanerva, Computing with 10,000-bit words
• Kanerva, Hyperdimensional Computing

Oct 16: Structural and Functional Connectomics (Tom Dean guest lecture)

• Lecture notes and slides

21,23,28 Oct

• Barlow, HB. Single units and sensation: A neuron doctrine for perceptual psychology? Perception, volume 1, pp. 371 -394 (1972)
• Foldiak, P. Forming sparse representations by local anti-Hebbian learning. Biol. Cybern. 64, 165-170 (1990).
• Olshausen BA, Field DJ. Emergence of simple-cell receptive field properties by learning a sparse code for natural images, Nature, 381: 607-609. (1996)

Additional readings:

• Rozell, Johnson, Baraniuk, Olshausen. Sparse Coding via Thresholding and Local Competition in Neural Circuits, Neural Computation 20, 2526–2563 (2008).
• Zylberberg, Murphy, DeWeese, A sparse coding model with synaptically local plasticity and spiking neurons can account for the diverse shapes of V1 simple cell receptive fields., PLOS Computational Biology, 7, (2011). (sparse coding with spiking neurons)
• Olshausen Sparse coding of time-varying natural images, ICIP 2003. (convolution sparse coding of video)
• Smith E, Lewicki MS. Efficient auditory coding, Nature Vol 439 (2006). (convolution sparse coding of sound)

30 Oct, 4 Nov

• HKP chapter 9, DA chapter 8
• Ocular dominance column development: Analysis and simulation by Miller, Keller and Stryker.
• A dimension reduction framework for understanding cortical maps by R. Durbin and G. Mitchison.
• The cortical column: a structure without a function by Jonathan C. Horton and Daniel L. Adams

Optional:

• Gary Blasdel, Orientation selectivity, preference, and continuity in monkey striate cortex., J Neurosci, 1992. Another source of many of nice images are in the galleries on Amiram Grinvald's site: [1]
• From Clay Reid's lab, Functional imaging with cellular resolution reveals precise micro-architecture in visual cortex. Make sure you look at the supplementary material and videos on their web site (seems partly broken) [2].

Re-organization in response to cortical lesions:

• Gilbert & Wiesel (1992), Receptive Field Dynamics in Adult Primary Visual Cortex
• Pettet & Gilbert (1992), Dynamic changes in receptive-field size in cat primary visual cortex

6 Nov

• A Global Geometric Framework for Nonlinear Dimensionality Reduction , Tenenbaum et al., Science 2000.
• Nonlinear Dimensionality Reduction by Locally Linear Embedding, Roweis and Saul, Science 2000.
• On the Local Behavior of Spaces of Natural Images, Carlsson et al., Int J Comput Vis (2008) 76: 1–12.

Additional reading:

• Adaptation to natural facial categories, Michael A. Webster, Daniel Kaping, Yoko Mizokami & Paul Duhamel, Nature, 2004.
• Prototype-referenced shape encoding revealed by high-level aftereffects, David A. Leopold, Alice J. O’Toole, Thomas Vetter and Volker Blanz, Nature, 2001.
• A Morphable Model For The Synthesis Of 3D Faces, Blanz & Vetter 1999.
• Matthew B. Thompson's web page on flashed face distortion effect

6, 13 Nov

• Marr-Poggio stereo algorithm paper
• Kechen Zhang paper on bump circuits
• Olshausen, Anderson & Van Essen, dynamic routing circuit model

13,18 Nov

• HKP Chapter 7, sections 7.2, 7.3
• Jaeger, echo state networks
• Alex Graves' thesis, see Chapter 4
• Sussillo, dynamical systems in neuroscience

20,25 Nov

• HKP chapter 7 (sec. 7.1),DJCM chapter 1-3, 20-24,41,43, DA chapter 10
• Olshausen (2014) Perception as an Inference Problem
• A probability primer
• Bayesian probability theory and generative models
• Mixture of Gaussians model
• T.J. Loredo, From Laplace to supernova SN1987A: Bayesian inference in astrophysics

2 Dec

• Crick and Mitchison theory on 'unlearning' during sleep - paper
• Application of Boltzmann machines to neural data analysis:
  
  • E. Schneidman, M.J. Berry, R. Segev and W. Bialek,Weak pairwise correlations imply strongly correlated network states in a neural population, Nature 4400 (7087) (2006), pp. 1007-1012.
    
  • J. Shlens, G.D. Field, J.L. Gauthier, M.I. Grivich, D. Petrusca, A. Sher, A.M. Litke and E.J. Chichilnisky, The structure of multi-neuron firing patterns in primate retina, J Neurosci 260 (32) (2006), pp. 8254-8266.
    
  • U. Koster, J. Sohl-Dickstein, C.M. Gray, B.A. Olshausen, Modeling higher-order correlations within Cortical Microcolumns, PLOS Computational Biology, July 2014.
• Kalman filter
  • Robbie Jacobs' notes on Kalman filter
  • kalman.m demo script
  • Greg Welch's tutorial on Kalman filter
  • Dynamic texture models
  • Kevin Murphy's HMM tutorial

4 Dec

• DA Chapter 10
• Information theory primer
• Sparse coding and ICA handout
• Jascha Sohl-Dickstein, The Natural Gradient by Analogy to Signal Whitening, and Recipes and Tricks for its Use
• Jascha Sohl-Dickstein, Natural gradient cookbook
• Bell & Sejnowski, An Information-Maximization Approach to Blind Separation and Blind Deconvolution, Neural Comp, 1995.
• Simoncelli, Olshausen. Natural Image Statistics and Neural Representation, Annu. Rev. Neurosci. 2001. 24:1193–216.
• van Hateren & Ruderman Independent component analysis of natural image sequences, Proc. R. Soc. Lond. B (1998) 265. (blocked sparse coding/ICA of video)
• Hyvarinen, Hoyer, Inki, Topographic Independent Component Analysis, Neural Comp, 2001.
• Karklin & Lewicki paper on Learning Higher-Order Structure in Natural Images, Network 2003.
• Shao & Cottrell paper on Recursive ICA, NIPS 2006.

9 Dec

Kalman filter:

• Robbie Jacobs' notes on Kalman filter
• kalman.m demo script
• Greg Welch's tutorial on Kalman filter
• Dynamic texture models
• Kevin Murphy's HMM tutorial

Spiking neurons:

• DA chapters 1-4, 5.4
• Karklin & Simoncelli, Efficient coding of natural images with a population of noisy Linear-Nonlinear neurons, NIPS 2011.
• Chris Eliasmith, Charlie Anderson, Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems, MIT Press, 2004. (Chapter 4 will be emailed to the class.)
• Softky and Koch, The Highly Irregular Firing of Cortical Cells Is Inconsistent with Temporal Integration of Random EPSPs, J Neuroscience, January 1993, 13(1):334-350.
• Mainen and Sejnowski, Reliability of Spike Timing in Neocortical Neurons, Science, Vol 268, 6 June 1995.
• Shadlen and Newsome, Noise, neural codes and cortical organization, Curr Opin in Neur, 1994, 4:569-579.
• Shadlen and Newsom, Is there a signal in the noise?, Current Opin in Neur, 1995, 5:248-250.
• Softky, Simple codes versus efficient codes, Current Opin in Neuro, 1995, 5:239-247.
• Izhikevich, Simple model of spiking neurons, IEEE Trans Neur Networks, 14(6):2003.
• Izhikevich, Which Model to Use for Cortical Spiking Neurons?, IEEE Trans Neur Networks, 15(5):2004.