By applying the Stress Process Model to examine the characteristics of people with dementia and their spousal caregivers, this study aims to identify the potential risk and mitigating factors of psychosocial adjustment among the spousal caregivers. We recruited 80 care recipient-caregiver couples in Hong Kong and examined the relationships of socio-economic, care recipient’s, and caregiver’s factors with spousal caregivers’ psychosocial adjustment. It was found that care recipients’ cognitive functions were associated with caregivers’ reported frequencies of their behavioral and psychological symptoms of dementia (BPSD) (r = .30, p = .008), and caregivers’ perceived burden of caregiving (r = -.54, p = .008). Caregivers’ quality of life was associated with their perceived caregiving burden (r = -.82, p = .001) and self-efficacy (r = .32, p = .001). Upon further examining the caregiving model with these parameters, a significant multivariate general linear model was found with (F (1, 12) = 13.06, p = .001, partial eta square = .70, observed power = .99). Moreover, female caregivers reported higher sense of caregiving stress and poorer quality of life than male caregivers. This study found that the cognitive functions and BPSD of care recipients with dementia and perceived level of caregiving burden are strongly associated with degrees of psychosocial adjustment among their spousal caregivers. The self-perceived caregiving role in a family is also a possible confounding factor contributing to the perceived caregiving burden. To support in-home caregiving of people with dementia, strategies to empower spousal caregivers to execute their caregiving roles are recommended.