Background From a young age the typical development of social functioning relies upon the allocation of attention to socially relevant information, which in turn allows experience at processing such information and thus enhances social cognition. As such, research has attempted to identify the developmental processes that are derailed in some neuro-developmental disorders that impact upon social functioning. Williams syndrome (WS) and autism are disorders of development that are characterized by atypical yet divergent social phenotypes and atypicalities of attention to people. Methods We used eye tracking to explore how individuals with WS and autism attended to, and subsequently interpreted, an actor’s eye gaze cue within a social scene. Images were presented for 3 seconds, initially with an instruction simply to look at the picture. The images were then shown again, with the participant asked to identify the object being looked at. Allocation of eye gaze in each condition was analyzed by analysis of variance and accuracy of identification was compared with t tests. Results Participants with WS allocated more gaze time to face and eyes than their matched controls, both with and without being asked to identify the item being looked at; while participants with autism spent less time on face and eyes in both conditions. When cued to follow gaze, participants with WS increased gaze to the correct targets; those with autism looked more at the face and eyes but did not increase gaze to the correct targets, while continuing to look much more than their controls at implausible targets. Both groups identified fewer objects than their controls. Conclusions The atypicalities found are likely to be entwined with the deficits shown in interpreting social cognitive cues from the images. WS and autism are characterized by atypicalities of social attention that impact upon socio-cognitive expertise, but, importantly, the type of atypicality is syndrome specific.