It has been suggested that male achievement in sports and athletics is correlated with a putative measure of prenatal testosterone the 2nd to 4th digit ratio (2D:4D). It is not known whether this association also extends to females, or whether the association results from an effect of testosterone on behavior (such as exercise frequency) or on physical fitness. Here, we report for the first time data from two studies which consider associations between 2D:4D and physical fitness in females in addition to males: Study I—in a sample of teenage boys (n = 114) and girls (n = 175), their ‘physical education grade’ was negatively associated with 2D:4D of the right hand (boys), and right and left hand (girls), and Study II—among a sample of young men (n = 102) and women (n = 77), a composite measure of physical fitness was negatively related to right hand 2D:4D in men and left hand 2D:4D in women. We conclude that 2D:4D is negatively related to physical fitness in both men and women. In Study II, there was evidence that the relationship between physical fitness and 2D:4D in men was mediated through an association with exercise frequency. Thus, 2D:4D in males may be a negative correlate of frequent exercise which then relates to achievement in sports and athletics.