Quantitative analysis at local and national level: getting it right and why it matters

The formulation of evidence-based policy necessitates rigorous, objective evaluation of policy initiatives and, consequently, there has been a significant growth in evaluation of social policy over the last ten years. Alongside this, there is a recognition that the application of new policy initiatives needs to be flexible in order to be relevant to local populations. As a result, pilots and pathfinders are encouraged to undertake local evaluations in addition to national evaluations commissioned by central government. These dual evaluations are seen as a vehicle to provide evidence on effectiveness whilst accommodating heterogeneity of needs and provision. We suggest that without clear delineation of roles, dual evaluations are inefficient, likely to put additional pressure on busy practitioners (and the recipients of new services) to comply with varying data demands, and present policy makers with confusing messages. In this article we focus on the potential for local and national evaluations to reach different conclusions by demonstrating how a simplistic application of quantitative techniques at local level can lead to inappropriate conclusions which contradict national findings. We make a number of recommendations that might facilitate better coordination of local and national evaluations.