Prosopagnosia is an impairment of the ability to process visual information derived from faces. A number of models of face processing have been proposed, using methods and models developed in cognitive psychology to explain the different forms of prosopagnosia. The proposed subcomponents of these models are based almost entirely on behavioural data from prosopagnosic human subjects. This paper reviews the physiological data on face processing, primarily obtained from single-cell recording from neurons in nonhuman primates, and relates this to a model of cognitive face processing. This allows localisation of the different processing modules in functional structures within the brain of higher primates, including man.