Caregivers of adolescents with autism spectrum disorder commonly experience stigma. However, how stigma influences social interactions of adolescents with autism spectrum disorder is unclear. We aimed to explore the impact of caregiver stigma on real-life social experiences of Taiwanese adolescents with autism spectrum disorder. In the context of everyday activities, 76 adolescents with autism spectrum disorder who were not intellectually disabled (69 males, aged 10–16 years) carried a mobile device that prompted them 7 times, randomly, each day for 7 days to record with whom they were interacting, what they perceived, and how they felt about the interactions. Caregivers completed the Affiliate Stigma Scale to measure caregiver stigma. Multilevel analyses revealed that participants whose caregivers perceived high levels of stigma were more likely than those whose caregivers experienced less stigma to interact with family members and less likely to be interested in interacting with people at school. However, those participants also experienced more anxiety while interacting with family members. The findings shed light on ways that caregiver stigma impacted the social experiences of adolescents with autism spectrum disorder and suggest that, in promoting social participation for adolescents with autism spectrum disorder, researchers and service providers must support caregivers to manage stigma.