Blurring the boundaries between synthesis and evaluation. A customized realist evaluative synthesis into adolescent risk behavior prevention

Realist methodologies have been increasingly advocated for the investigation of complex social issues. Public health programs, such as those designed to prevent adolescent risk behavior, are typically considered complex. In conducting a realist review of the empirical literature relating to such programs, we encountered several challenges, including (a) an overabundance of empirical evidence, (b) a problematic level of heterogeneity within and between methodological approaches, (c) discrepancies between theoretical underpinnings and program operationalization, (d) homogeneity of program outcomes, with very little variation in program effectiveness, and (d) a paucity of description relating to content and process. To overcome these challenges, we developed a customized approach to realist evidence synthesis, drawing on the VICTORE (Volition, Implementation, Contexts, Time, Outcomes, Rivalry, and Emergence) complexity checklist and incorporating stakeholder engagement as primary data to achieve greater depth of understanding relating to contextual and mechanistic factors, and the complex interactions between them. Here we discuss the benefits of this adapted methodology alongside an overview of the research through which the methodology was developed. A key finding from this research was that combining the complexity checklist with primary data from stakeholder engagement enabled us to systematically interrogate the data across data sources, uncovering and evidencing mechanisms which may otherwise have remained hidden, giving greater ontological depth to our research findings. This paper builds on key methodological developments in realist research, demonstrating how realist methodologies can be customized to overcome challenges in developing and refining program theory from the literature, and contributes to the broader literature of innovative approaches to realist research. Highlights: Published reporting standards for realist review provide a set of guiding principles for conducting realist research. However, these are not recommended to be used in a prescriptive sense, and customization of the methodology to account for potential idiosyncrasies within a specific evidence base is accepted. A small number of papers within the existing literature have used each of the two key adaptions discussed within this study, though reasons for doing so have not been considered in any great depth. Furthermore, combining both of these adaptions to take an evaluative approach to realist synthesis is novel to this work and lends greater ontological depth to the research findings than may otherwise have been achieved. This study builds on key methodological developments in realist research, demonstrating how realist methodologies can be customized to overcome challenges in developing and refining program theory from the literature, and contributes to the broader literature of innovative approaches to realist research.