Mission and Research

Invariance

Our mental experience suggests that the brain encodes and manipulates 'objects' and their relationships, but there is no neural theory of how this is done. We recognize, for example, a cup regardless of its location, orientation, size, or other variations such as lighting and partial occlusion. How do brain networks recognize a cup despite these complicated variations in the image data? How is the invariant part ('cup-ness') encoded separately from the variant part?

This is called the invariance problem. It is a 'holy grail' problem of the computer vision community, and we aim to tackle it by fortifying our learning algorithms with insights from the mathematics surrounding the concept of invariance. Invariance may also be seen in motor scenarios, cups being a class of things that we can drink from (what J.J.Gibson called an affordance).

As we ascend the cortical hierarchy from area V1, we find increasingly invariant forms of coding. It is our goal to understand these forms of coding and how they may be learned from natural data. A modest success in this direction is that 'complex cell' receptive fields (oriented and localized contrast sensitive neurons which are invariant to spatial phase) can be learned in this way.

Learning

Learning is arguably the central problem in theoretical neuroscience. It is possible that other problems such as the understanding of representations, network dynamics and circuit function may ultimately be best understood through understanding the learning processes that, together with the action of the genome, produce these phenomena.

To solve the problem, it will be necessary to combine the best ideas from statistical machine learning with the cleverest plasticity studies at the synaptic and network level. Utilizing the intersection of these two forms of knowledge greatly constrains the search space. This is necessary since twenty years of abstract neural network theory have done little more than, for example, self-organizing the correct forms for receptive fields in area V1 of cortex, using a single layer feedforward network of `connectionist' neurons, and an ensemble of natural images.

Our efforts have focused on unsupervised learning using sparse coding principles and information theory. It is controversial whether learning in the brain is unsupervised or reinforcement-based. Although reinforcement undeniably flows from subcortical structures, the framework of reinforcement learning requires a hard-coded, or 'given', reward signal that is external to the operation of the network. This is not the case in the brain considered as a whole.

Breaking the learning problem down, we are focusing on the attempt to learn invariant forms of coding from sensory stimuli (and later, we hope, sensorimotor scenarios) and on an attempt to explain why changes in neural connection strengths depend on the relative timings of incoming and outgoing spikes.

Since the learning algorithms we develop are general multivariate data analysis algorithms, we also use them to analyse (or data-mine) neurophysiological recordings from, for example, EEG, MEG, fMRI and optical imaging techniques, helping neurophysiologists to remove noise and isolate components of brain activity relevant to sensory, perceptual and motor phenomena.

Associative memory

W. James, F. Hayek and D. O. Hebb proposed theories of memory and mental association involving distributed neural representations and synaptic plasticity. Neuronal associative memories are abstract neural networks that implement the basic mechanisms of learning and association as postulated in Hebb's theory. We believe that principles of associative memories are important in tackling central problems in theoretical neuroscience:

1. Invariant sensory processing
   A recent model of invariant sensory processing demonstrates that a memory-based strategy is applicable to real images (Map seeking circuits. Arathorn, 2002). We are interested in studying the mathematical basis of memory based models of invariant sensory processing.
2. Formation of compositional memories
   In models of cognition it is crucial that concepts can be compositional and multi-facetted. Holographic representations (Plate; 1994, 2001), spatter coding (Kanerva; 1994) and vector-symbolic architectures (Gayler; 1998) are methods to form compositional distributed representations. All these methods rely critically on associative memories. So far, the existing methods do not scale up to real-world problems. We are interested in designing models with efficient sparse associative memories that scale up to real-world domains.
3. Communication in cortico-cortical networks
   The brain is organized in functionally specialized regions connected by a cortico-cortical connections with small network properties (Strogatz). We are interested in extending the theory of associative memory to count the number of possible functional networks in anatomically constrained networks. Potentially, this analysis will reveal important conditions of cortico-cortical information processing, in particular, the properties of distributed representations in local regions that yield high numbers of possible functional networks and thus provide high flexibility in the formation of functional networks.
4. Hierarchical memory models
   Ultimately, we are interested in designing memory systems that combine the results of the both previous studies, that is, compositional memory representations that can be communicated in a structured neuronal network. In such networks hierarchies will be defined by the degree of convergence from different modalities (regions with different functional specialisation).
5. Unsupervised/supervised learning of memories (role of neuromodulatory systems)
   In standard models of associative memory, the selection of memories to be stored is done externally, in a supervised fashion. We are interested in models that screen continuous input and decide internally what memories to store.

Exploratory data analysis

Brain imaging techniques, such as functional MRI, EEG, open macroscopic windows on processes in the working brain. These methods yield high dimensional data sets that are organized in space (brain coordinates) and time. The current analysis methods extract interpretable images from the data but these methods are far from harvesting the full richness of the measured data. We are interested in developing exploratory analysis methods to assess the statistical properties in the joint data set combining imaging data and behavior/stimulus data.

Single-cell/network/biophysical models

There are two approaches we are following at the physiological level.

The first is modeling of physiological processes. We address how the response properties of neurons, such as synaptic integration and receptive fields, arise based on experimental observations such as the biophysical properties of single neurons, connectivity in the cortex, and in vivo recordings. An important aspect of a physiological neural model is, besides replicating the data it is based on, to come with predictions for responses, e.g. to novel stimuli, that can be verified or rejected in physiological experiments. This way theory can aid to guide the direction of experiments to gain greater understanding of the brain.

The second approach is to bring neural network ideas from machine learning down to the membrane level, so that phenomena such as Spike-Timing Dependent Plasticity (the most striking phenomenon in synaptic learning) may be understood as information theoretic or probabilistic optimizations. A massive amount of data has accumulated on the molecular basis of neural plasticity. The time is ripe to integrate it in a theoretical framework. If this framework is correct, we will be able to self-organise networks of spiking neurons, facilitating further studies of sensory coding, circuit dynamics, and the function of associative and sensory-motor loops.

At the Redwood Center we apply theoretical ideas at a range of levels of physiological modeling, from single cell models addressing properties of dendritic summation of synaptic input to large network models looking at responses in the primary visual cortex (V1).

Multiscale interactions and oscillations

Brain activity can be described at various levels of resolution. The neuron level, on which single neurons constitute the fundamental computational units, is the most common level for theories of sensory perception. However, some theories of plasticity and learning are formulated on the level of individual synapses. Further, theories of cognitive functions like decision making and attention operate on the level of neuron populations.

Both, the neuron level and the population level are directly accessible to electro-physiological measurements. The activity in single neurons can be recorded with single or multiple electrodes. The activity of single neurons is best described as point processes in time that correspond to the spikes of individual cells. The population activity of many neurons can be recorded as local field potentials or as activity in the Electro-Corticogram (ECoG) or in the Electro-Encephalogram (EEG). The population activity is a continuous signal extended in space and time and often has oscillatory properties.

In addition to studies that investigate neural activity on an individual level, it is crucial to understand how different levels interact: We would like to understand how spike activity of individual neurons forms population activity, and how in turn the population activity influences the response properties of individual neurons.

There is an intriguing parallel between multiple nested levels of brain activity and the multi-scale structure of sensory data. It is conceivable that different structural scales in sensory data are processed not only at different levels of the cortical hierarchy but also at different levels of brain activity.

Active perception and sensorimotor loops

Perception is an active process. During natural vision, for example, our eyes are constantly moving even when we fixate an object. In addition, active, internal processes in the brain, such as attention, influence the processing of sensory information. Thus, action and perception affect each other tightly, which is often called the sensorimotor loop.

If perception and action are coupled, the brain must learn to distinguish whether a change in a sensory input reflects a change in the outside world or is a result of our own action. Theories of active perception claim that our ability to trace how own actions can influence sensory perception is the basis of object recognition. Objects correspond to invariances in sensorimotor space rather than in pure sensory space. Such invariances in sensorimotor space are called sensorimotor contingencies. We are interested in the learning of sensorimotor contingencies and how they are used during active perception.