Natural images contain characteristic statistical regularities that set
them apart from purely random images. Understanding what these regularities
are can enable natural images to be coded more efficiently. In this paper,
we describe some of the forms of structure that are contained in natural
images, and we show how these are related to the response properties of
neurons at early stages of the visual system. Many of the important forms
of structure require higher-order (i.e., more than linear, pairwise) statistics
to characterize, which makes models based on linear hebbian learning, or
principal components analysis, inappropriate for finding efficient codes
for natural images. We suggest that a good objective for an efficient coding
of natural scenes is to maximize the sparseness of the representation,
and we show that a network that learns sparse codes of natural scenes succeeds
in developing localized, oriented, bandpass receptive fields similar to
those in the primate striate cortex.