Sebastian Musslick
Princeton Neuroscience Institute (Princeton University)

Parallel Processing Capability Versus Efficiency of Representation in Neural Network Architectures

Wednesday 16th of November 2016 at 12:00pm
560 Evans

One of the most salient and well-recognized features of human goal-directed behavior is our limited ability to conduct multiple demanding tasks at once. Why is this? Some have suggested it reflects metabolic limitations, or structural ones. However, both explanations are unlikely. The brain routinely demonstrates the ability to carry out a multitude of processes in an enduring and parallel manner (walking, breathing, listening). Why, in contrast, is its capacity for allocating attention to control-demanding tasks - such a critical and powerful function - so limited? In the first part of my talk I will describe a computational framework that explains limitations of parallel processing in neural network architectures as the result of cross-talk between shared task representations. Using graph-theoretic analyses we show that the parallel processing (multitasking) capability of two-layer networks drops precipitously as a function of task pathway overlap, and scales highly sublinearly with network size. I will describe how this analysis can be applied to task representations encoded in neural networks or neuroimaging data, and show how it can be used to predict both concurrent and sequential multitasking performance in trained neural networks based on single task representations. Our results suggest that maximal parallel processing performance is achieved by segregating task pathways, by separating the representations on which they rely. However, there is a countervailing pressure for pathways to intersect: the re-use of representations to facilitate learning of new tasks. In the second part of my talk I will demonstrate a tradeoff between learning efficiency and parallel processing capability in neural networks. It can be shown that weight priors on learned task similarity improve learning speed and generalization but lead to strong constraints on parallel processing capability. These findings will be contrasted with an ongoing behavioral study by assessing learning and multitasking performance of human subjects across tasks with varying degrees of feature-overlap.

Join Email List

You can subscribe to our weekly seminar email list by sending an email to majordomo@lists.berkeley.edu that contains the words subscribe redwood in the body of the message.
(Note: The subject line can be arbitrary and will be ignored)