Conceptualizing and Measuring Sustainability of Prevention Programs, Policies, and Practices

Overview

Journal Transl Behav Med

Publisher Oxford University Press

Specialties General Medicine
Public Health
Social Sciences

Date 2019 Nov 26

PMID 31764968

Citations 27

Authors

Lawrence A Palinkas

Suzanne E Spear

Sapna J Mendon

Juan Villamar

Charles Reynolds

Costella D Green

Charlotte Olson

Audrey Adade

C Hendricks Brown

Affiliations

Soon will be listed here.

Abstract

A large knowledge gap exists regarding the measurement of sustainability of evidence-based prevention programs for mental and behavioral health. We interviewed 45 representatives of 10 grantees and 9 program officers within 4 Substance Abuse and Mental Health Services Administration prevention grant initiatives to identify experiences with implementation and sustainability barriers and facilitators; what "sustainability" means and what it will take to sustain their programs; and which Consolidated Framework for Implementation Research (CFIR) elements are important for sustainability. Lists of sustainability determinants and outcomes were then compiled from each data set and compared with one another. Analysis of themes from interviews and free lists revealed considerable overlap between sustainability determinants and outcomes. Four sustainability elements were identified by all three data sets (ongoing coalitions, collaborations, and networks and partnerships; infrastructure and capacity to support sustainability; community need for program; and ongoing evaluation of performance and outcomes), and 11 elements were identified by two of three data sets (availability of funding; consistency with organizational culture; evidence of positive outcomes; development of a plan for implementation and sustainment; presence of a champion; institutionalization and integration of program; institutional support and commitment; community buy-in and support; program continuity; supportive leadership; and opportunities for staff training). All but one of the CFIR domain elements (pressure from other states, tribes, or communities) were endorsed as important to sustainability by 50% or more of participants. It may be more important to implement a standardized process of eliciting determinants and outcomes of sustainability than to implement a single standardized instrument.

Citing Articles

Usability of a cross-system housing stability program for juvenile courts: A multimethod study of probation counselor perspectives.

Bishop A, Walker S, Baumgarten E, Vick K, Dean T Implement Res Pract. 2025; 6:26334895251319814.

PMID: 39991469 PMC: 11843703. DOI: 10.1177/26334895251319814.

Applying Implementation Science to Advance Electronic Health Record-Driven Learning Health Systems: Case Studies, Challenges, and Recommendations.

Trinkley K, Maw A, Huebner Torres C, Huebschmann A, Glasgow R J Med Internet Res. 2024; 26:e55472.

PMID: 39374069 PMC: 11494259. DOI: 10.2196/55472.

Providers' perspectives of barriers and facilitators to scale-up of mental health care in the public health delivery system of Mozambique: a qualitative inquiry.

Mootz J, de Vos L, Stockton M, Sweetland A, Kann B, Seijo C BMC Health Serv Res. 2024; 24(1):1138.

PMID: 39334160 PMC: 11428441. DOI: 10.1186/s12913-024-11594-9.

A mixed-methods, theory-driven assessment of the sustainability of a multi-sectoral preventive intervention for South Asian Americans at risk for cardiovascular disease.

Vu M, Nedunchezhian S, Lancki N, Spring B, Brown C, Kandula N Implement Sci Commun. 2024; 5(1):89.

PMID: 39267181 PMC: 11396489. DOI: 10.1186/s43058-024-00626-4.

Virtual Reality Job Interview Training in Transition Services: Results of a Single-Arm, Noncontrolled Effectiveness-Implementation Hybrid Trial.

Smith M, Smith J, Jordan N, Sherwood K, McRobert E, Ross B J Spec Educ Technol. 2024; 36(1):3-17.

PMID: 38911489 PMC: 11192452. DOI: 10.1177/0162643420960093.

References

Wiltsey Stirman S, Kimberly J, Cook N, Calloway A, Castro F, Charns M . The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012; 7:17. PMC: 3317864. DOI: 10.1186/1748-5908-7-17. View

Saldana L . The stages of implementation completion for evidence-based practice: protocol for a mixed methods study. Implement Sci. 2014; 9(1):43. PMC: 4234147. DOI: 10.1186/1748-5908-9-43. View

Scheirer M, Dearing J . An agenda for research on the sustainability of public health programs. Am J Public Health. 2011; 101(11):2059-67. PMC: 3222409. DOI: 10.2105/AJPH.2011.300193. View

Walugembe D, Sibbald S, Le Ber M, Kothari A . Sustainability of public health interventions: where are the gaps?. Health Res Policy Syst. 2019; 17(1):8. PMC: 6334403. DOI: 10.1186/s12961-018-0405-y. View

Hodge L, Turner K, Sanders M, Filus A . Sustained Implementation Support Scale: Validation of a Measure of Program Characteristics and Workplace Functioning for Sustained Program Implementation. J Behav Health Serv Res. 2016; 44(3):442-464. DOI: 10.1007/s11414-016-9505-z. View

Goldston D, Walrath C, McKeon R, Puddy R, Lubell K, Potter L . The Garrett Lee Smith memorial suicide prevention program. Suicide Life Threat Behav. 2010; 40(3):245-56. PMC: 3107991. DOI: 10.1521/suli.2010.40.3.245. View

Shelton R, Lee M . Sustaining Evidence-Based Interventions and Policies: Recent Innovations and Future Directions in Implementation Science. Am J Public Health. 2019; 109(S2):S132-S134. PMC: 6383970. DOI: 10.2105/AJPH.2018.304913. View

Kellam S, Brown C, Poduska J, Ialongo N, Wang W, Toyinbo P . Effects of a universal classroom behavior management program in first and second grades on young adult behavioral, psychiatric, and social outcomes. Drug Alcohol Depend. 2008; 95 Suppl 1:S5-S28. PMC: 2512256. DOI: 10.1016/j.drugalcdep.2008.01.004. View

Pluye P, Potvin L, Denis J, Pelletier J . Program sustainability: focus on organizational routines. Health Promot Int. 2004; 19(4):489-500. DOI: 10.1093/heapro/dah411. View

10.

Saldana L, Chamberlain P, Wang W, Brown C . Predicting program start-up using the stages of implementation measure. Adm Policy Ment Health. 2011; 39(6):419-25. PMC: 3212640. DOI: 10.1007/s10488-011-0363-y. View

11.

Proctor E, Luke D, Calhoun A, McMillen C, Brownson R, McCrary S . Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement Sci. 2015; 10:88. PMC: 4494699. DOI: 10.1186/s13012-015-0274-5. View

12.

Palinkas L, Spear S, Mendon S, Villamar J, Valente T, Chou C . Measuring sustainment of prevention programs and initiatives: a study protocol. Implement Sci. 2016; 11:95. PMC: 4947241. DOI: 10.1186/s13012-016-0467-6. View

13.

Cooper B, Bumbarger B, Moore J . Sustaining evidence-based prevention programs: correlates in a large-scale dissemination initiative. Prev Sci. 2013; 16(1):145-57. DOI: 10.1007/s11121-013-0427-1. View

14.

Luke D, Calhoun A, Robichaux C, Elliott M, Moreland-Russell S . The Program Sustainability Assessment Tool: a new instrument for public health programs. Prev Chronic Dis. 2014; 11:130184. PMC: 3900326. DOI: 10.5888/pcd11.130184. View

15.

Rhew I, Brown E, Hawkins J, Briney J . Sustained effects of the Communities That Care system on prevention service system transformation. Am J Public Health. 2012; 103(3):529-35. PMC: 3673484. DOI: 10.2105/AJPH.2011.300567. View

16.

Nilsen P . Making sense of implementation theories, models and frameworks. Implement Sci. 2015; 10:53. PMC: 4406164. DOI: 10.1186/s13012-015-0242-0. View

17.

Feinberg M, Bontempo D, Greenberg M . Predictors and level of sustainability of community prevention coalitions. Am J Prev Med. 2008; 34(6):495-501. DOI: 10.1016/j.amepre.2008.01.030. View

18.

Gloppen K, Arthur M, Hawkins J, Shapiro V . Sustainability of the Communities That Care prevention system by coalitions participating in the Community Youth Development Study. J Adolesc Health. 2012; 51(3):259-64. PMC: 3428591. DOI: 10.1016/j.jadohealth.2011.12.018. View

19.

Johnson A, Moore J, Chambers D, Rup J, Dinyarian C, Straus S . How do researchers conceptualize and plan for the sustainability of their NIH R01 implementation projects?. Implement Sci. 2019; 14(1):50. PMC: 6506963. DOI: 10.1186/s13012-019-0895-1. View

20.

Han S, Weiss B . Sustainability of teacher implementation of school-based mental health programs. J Abnorm Child Psychol. 2005; 33(6):665-79. DOI: 10.1007/s10802-005-7646-2. View