Jascha Sohl-Dickstein

I am a graduate student in the Redwood Center for Theoretical Neuroscience, at University of California, Berkeley. I am a member of Bruno Olshausen's lab.

I am interested in how we learn to perceive the world. That is, how we start receiving sensory input (seeing, hearing, smelling, touching, etc) ... and after a while figure out the way the world works. We learn that the light intensity on adjacent patches of retina is correlated, that the world is made up of edges and surfaces, how occlusion works, that objects exist, that if you drop them they fall and make a noise and hurt... We seem to infer most of human scale physics from examples of sensory input.

How this unsupervised learning problem is solved - how we learn the structure inherent in the world just by experiencing examples of it - is not understood. This is the problem I am interested in tackling.

There are two large (known) parts to this problem. The first is the design of models which are flexible enough to describe *anything* without being unwieldy (greater flexibility frequently comes at the cost of an explosion in the number of parameters and/or more computationally costly implementation). The second is practically training these models once they've been designed - estimating their parameters for a particular dataset (Almost any model you can write down is impractical to exactly evaluate - a factor called the "partition function", coming from the constraint that probabilities of all states must sum to 1, makes it intractable to compute. This inability to exactly evaluate the model makes training exceedingly difficult.). The majority of my work has been on parameter estimation in traditionally intractable models.

Current projects

I am working with Peter Battaglino and Michael DeWeese on a technique for parameter estimation in probabilistic models with intractable partition functions, involving minimization of probability flows. See the arXiv pre/e-print.

I am working with Jimmy Wang and Bruno Olshausen to build a Lie algebraic model of the transformations which occur in natural video. See Jimmy's web page, or a poster pdf.

I am working on my own to build probabilistic models out of unstructured recurrent neural networks, and train them on natural stimuli. Nothing to show here yet, but I am eager to discuss.

Relevant publications

J Sohl-Dickstein, P Battaglino, M DeWeese. Minimum probability flow learning. (2009) http://arxiv.org/abs/0906.4779

C Abbey, J Sohl-Dickstein, B Olshausen. Higher-order scene statistics of breast images. Proceedings of SPIE (2009) http://link.aip.org/link/?PSISDG/7263/726317/1

POSTER - J Sohl-Dickstein, B Olshausen. Learning in energy based models via score matching. Cosyne (2007) - this (dense!) poster introduces a spatial derivation of score matching, applies it to learning in a Field of Experts model, and then extends Field of Experts to work with heterogeneous experts (to form a "tapestry of experts").

Notes

I am intending to place short informal documents here on a variety of topics which interest me and I have something to say about. If any of the following sound interesting to you, pester me and they will appear more quickly.

• Natural gradients made quick and dirty
• A log bound on the growth of intelligence with system size
• The field of experts model learns Gabors when trained via score matching
• For small time bins, generalized linear models (with exponential pointwise nonlinearities) and Boltzmann machines become equivalent
• How to construct a volume preserving recurrent network

Interests

If you know anything about the following, I would love to pick your brain:

• non-equilibrium statistical mechanics (Jaynes, Crooks, Jarzynski...)
• criticality
• compressive sensing (specifically - connections to information theory / manifolds)
• echo state networks / liquid state machines (specifically - what can we say about the classes of transformations the input undergoes)
• Ways to close the sensori-motor loop. (What's the objective function for the brain?)
• Brain-machine interfaces (especially - thoughts about software on the machine side which is able to adapt intelligently, and online, to brain output)

Papers from my previous life as a Martian

Kinch et al. Dust deposition on the Mars Exploration Rover Panoramic Camera (Pancam) calibration targets. Journal of Geophysical Research-Planets (2007)

Johnson et al. Radiative transfer modeling of dust-coated Pancam calibration target materials: Laboratory visible/near-infrared spectrogoniometry. J. Geophys. Res (2006)

Joseph et al. In-flight calibration and performance of the Mars Exploration Rover Panoramic Camera (Pancam) Instruments. J. Geophys. Res (2006)

Parker et al. Stratigraphy and sedimentology of a dry to wet eolian depositional system, Burns formation, Meridiani Planum, Mars. Earth and Planetary Science Letters (2005)

Soderblom et al. Pancam multispectral imaging results from the Opportunity rover at Meridiani Planum. Science (2004)

Soderblom et al. Pancam multispectral imaging results from the Spirit rover at Gusev crater. Science (2004)

Smith et al. Athena microscopic imager investigation. Journal of Geophysical Research-Planets (2003)

Bell et al. Hubble Space Telescope Imaging and Spectroscopy of Mars During 2001. American Geophysical Union (2001)