Predictive validity of the START for unauthorised leave and substance abuse in a secure mental health setting: A pseudo-prospective cohort study

Background: Risk assessment and management is central to the nursing role in forensic mental health settings. The Short Term Assessment of Risk and Treatability (START) aims to support assessment through identification of risk and protective factors. It has demonstrated predictive validity for aggression; it also aims to aid risk assessment for unauthorised leave and substance abuse where its performance is relatively untested. Objectives: To test the predictive validity of the START for unauthorised leave and substance abuse. Design: A naturalistic, pseudo-prospective cohort study. Settings: Four centres of a large UK provider of secure inpatient mental health services. Participants: Inpatients resident between May 2011 and October 2013 who remained in the service for 3-months following assessment with the START by their clinical team. Exclusion criteria were missing assessment data in excess of prorating guidelines. Of 900 eligible patients 73 were excluded leaving a final sample size of n = 827 (response rate 91.9%). Mean age was 38.5 years (SD = 16.7); most participants (72.2%) were male; common diagnoses were schizophrenia-type disorders, personality disorders, organic disorders, developmental disorders and intellectual disability. Methods: Routinely conducted START assessments were gathered. Subsequent incidents of substance abuse and unauthorised leave were coded independently. Positive and negative predictive values of low and elevated risk were calculated. Receiver Operating Characteristic analysis was conducted to ascertain the predictive accuracy of the assessments based on their sensitivity and specificity. Results: Patient-based rates of unauthorised leave (2.4%) and substance abuse (1.6%) were low. The positive and negative predictive values for unauthorised leave were 5.9% and 98.4%; and for substance abuse 8.1% and 99.0%. The START specific risk estimate for unauthorised leave predicted its associated outcome (Area under the curve = .659, p< .05, 95% CI .531, .786); the substance abuse risk estimate predicted its outcome with a large effect size (Area under the curve = .723, p< .01, 95% CI .568, .879). Conclusions: The study provides limited support for the START by demonstrating the predictive validity of its specific risk estimates for substance abuse and unauthorised leave. High negative predictive values suggest the tool may be of most utility in screening out low risk individuals from unnecessary restrictive interventions; very low positive predictive values suggest caution before implementing restrictive interventions in those rated at elevated risk. Researchers should investigate how multidisciplinary teams formulate risk assessments for these outcomes since they outperform the quantitative element of this tool.