Testosterone (T) has been argued to modulate mating and parenting behavior in many species, including humans. The role of T for these behaviors has been framed as the challenge hypothesis. Following this hypothesis, T should be positively associated with the number of opposite sex partners a male has. Indeed research in humans has shown that T is positively related to the number of opposite sex partners a young man has had. Here we test, in both men and women, whether this relationship extends to the lifetime number of sex partners. We also explored whether or not T was associated with current marital status, partnership status and whether or not the participant remarried. Using a large sample of elderly men and women (each sample n > 700), we show that T is positively and sizably associated with the number of opposite sex partners in men. When controlling for potential confounding variables such as educational attainment, age, BMI, ethnicity, specific use of a medication and time of sampling this effect remained. For women, the relationship between T and number of opposite sex partners was positive but did not prove to be robust. In both men and women there was no evidence for an association between T and current marital status and partnership status (being in a relationship or not). However, remarriage was positively associated with T, but only in males. Results are discussed with reference to the literature on T and sex partners, remarriage and more broadly the challenge hypothesis.