Assessing the Evaluability of Complex Public Health Interventions: Five Questions for Researchers, Funders, and Policymakers

Overview

Journal Milbank Q

Specialties Health Services
Public Health

Date 2011 Jun 17

PMID 21676021

Citations 41

Authors

David Ogilvie

Steven Cummins

Mark Petticrew

Martin White

Andy Jones

Kathryn Wheeler

Affiliations

Soon will be listed here.

Abstract

Context: Evidence to support government programs to improve public health often is weak. Recognition of this "knowledge gap" has led to calls for more and better evaluation, but decisions about priorities for evaluation also need to be addressed in regard to financial restraint.

Methods: Using England's Healthy Community Challenge Fund as a case study, this article presents a set of questions to stimulate and structure debate among researchers, funders, and policymakers and help make decisions about evaluation within and between complex public health interventions as they evolve from initial concept to dissemination of full-scale intervention packages.

Findings: This approach can be used to identify the types of knowledge that might be generated from any evaluation, given the strength of evidence available in response to each of five questions, and to support a more systematic consideration of resource allocation decisions, depending on the types of knowledge required.

Conclusions: The principles of this approach may be generalizable, and should be tested and refined for other complex public health and wider social interventions.

Citing Articles

Wellbeing Impact Study of High-Speed 2 (WISH2): Protocol for a mixed-methods examination of the impact of major transport infrastructure development on mental health and wellbeing.

Morley K, Hocking L, Saunders C, Bousfield J, Bostock J, Brimicombe J PLoS One. 2024; 19(2):e0298701.

PMID: 38422089 PMC: 10903902. DOI: 10.1371/journal.pone.0298701.

Prioritisation processes for programme implementation and evaluation in public health: A scoping review.

Atwal S, Schmider J, Buchberger B, Boshnakova A, Cook R, White A Front Public Health. 2023; 11:1106163.

PMID: 37050947 PMC: 10083497. DOI: 10.3389/fpubh.2023.1106163.

Exploring the potential impact of the proposed UK TV and online food advertising regulations: a concept mapping study.

Forde H, Boyland E, Scarborough P, Smith R, White M, Adams J BMJ Open. 2022; 12(6):e060302.

PMID: 35715182 PMC: 9207937. DOI: 10.1136/bmjopen-2021-060302.

Identifying priorities for research on financial risk protection to achieve universal health coverage: a scoping overview of reviews.

Bhatia D, Mishra S, Kirubarajan A, Yanful B, Allin S, Di Ruggiero E BMJ Open. 2022; 12(3):e052041.

PMID: 35264342 PMC: 8915291. DOI: 10.1136/bmjopen-2021-052041.

How should we evaluate sweetened beverage tax policies? A review of worldwide experience.

Ng S, Colchero M, White M BMC Public Health. 2021; 21(1):1941.

PMID: 34702248 PMC: 8546197. DOI: 10.1186/s12889-021-11984-2.

References

Hill A . THE ENVIRONMENT AND DISEASE: ASSOCIATION OR CAUSATION?. Proc R Soc Med. 1965; 58:295-300. PMC: 1898525. DOI: 10.1177/003591576505800503. View

Simmons R, Ogilvie D, Griffin S, Sargeant L . Applied public health research -- falling through the cracks?. BMC Public Health. 2009; 9:362. PMC: 2760531. DOI: 10.1186/1471-2458-9-362. View

Wilson P, Petticrew M, Calnan M, Nazareth I . Why promote the findings of single research studies?. BMJ. 2008; 336(7646):722. PMC: 2276286. DOI: 10.1136/bmj.39525.447361.94. View

Bond L, Craig P, Egan M, Skivington K, Thomson H . Evaluating complex interventions. Health improvement programmes: really too complex to evaluate?. BMJ. 2010; 340:c1332. DOI: 10.1136/bmj.c1332. View

Lorenc T, Brunton G, Oliver S, Oliver K, Oakley A . Attitudes to walking and cycling among children, young people and parents: a systematic review. J Epidemiol Community Health. 2008; 62(10):852-7. DOI: 10.1136/jech.2007.070250. View

Westley H . Thin living. BMJ. 2007; 335(7632):1236-7. PMC: 2137045. DOI: 10.1136/bmj.39409.451678.AD. View

Freedman B . Equipoise and the ethics of clinical research. N Engl J Med. 1987; 317(3):141-5. DOI: 10.1056/NEJM198707163170304. View

Mackenzie M, ODonnell C, Halliday E, Sridharan S, Platt S . Do health improvement programmes fit with MRC guidance on evaluating complex interventions?. BMJ. 2010; 340:c185. DOI: 10.1136/bmj.c185. View

Petticrew M, Whitehead M, Macintyre S, Graham H, Egan M . Evidence for public health policy on inequalities: 1: the reality according to policymakers. J Epidemiol Community Health. 2004; 58(10):811-6. PMC: 1763325. DOI: 10.1136/jech.2003.015289. View

10.

Hawe P, Shiell A, Riley T . Theorising interventions as events in systems. Am J Community Psychol. 2009; 43(3-4):267-76. DOI: 10.1007/s10464-009-9229-9. View

11.

Exworthy M . Policy to tackle the social determinants of health: using conceptual models to understand the policy process. Health Policy Plan. 2008; 23(5):318-27. DOI: 10.1093/heapol/czn022. View

12.

Leviton L, Khan L, Rog D, Dawkins N, Cotton D . Evaluability assessment to improve public health policies, programs, and practices. Annu Rev Public Health. 2010; 31:213-33. DOI: 10.1146/annurev.publhealth.012809.103625. View

13.

Ginexi E, Hilton T . What's next for translation research?. Eval Health Prof. 2006; 29(3):334-47. DOI: 10.1177/0163278706290409. View

14.

Thomson H, Hoskins R, Petticrew M, Ogilvie D, Craig N, Quinn T . Evaluating the health effects of social interventions. BMJ. 2004; 328(7434):282-5. PMC: 324463. DOI: 10.1136/bmj.328.7434.282. View

15.

Ogilvie D, Craig P, Griffin S, Macintyre S, Wareham N . A translational framework for public health research. BMC Public Health. 2009; 9:116. PMC: 2681470. DOI: 10.1186/1471-2458-9-116. View

16.

Sallis J, Story M, Lou D . Study designs and analytic strategies for environmental and policy research on obesity, physical activity, and diet: recommendations from a meeting of experts. Am J Prev Med. 2009; 36(2 Suppl):S72-7. PMC: 2694128. DOI: 10.1016/j.amepre.2008.10.006. View

17.

Shepperd S, Lewin S, Straus S, Clarke M, Eccles M, Fitzpatrick R . Can we systematically review studies that evaluate complex interventions?. PLoS Med. 2009; 6(8):e1000086. PMC: 2717209. DOI: 10.1371/journal.pmed.1000086. View

18.

Stamatakis E, Ekelund U, Wareham N . Temporal trends in physical activity in England: the Health Survey for England 1991 to 2004. Prev Med. 2007; 45(6):416-23. DOI: 10.1016/j.ypmed.2006.12.014. View