Recruiting and Retaining Youth and Young Adults in the Policy and Communication Evaluation (PACE) Vermont Study: Randomized Controlled Trial of Participant Compensation

Background: The standard approach for evaluating the effects of population-level substance use prevention efforts on youth and young adult perceptions and behaviors has been to compare outcomes across states using national surveillance data. Novel surveillance methods that follow individuals over shorter time intervals and capture awareness of substance use prevention policy and communication efforts may provide a stronger basis for their evaluation than annual cross-sectional studies.

Objective: This study aimed to identify a combination of strategies to recruit a sample of youth and young adults sufficiently representative of the Vermont population and determine how best to retain a web-based panel of youth and young adults over a 6-month period.

Methods: Eligible participants were Vermont residents aged 12 to 25 years who were willing to complete three 10 to 15-minute web-based surveys over a 6-month period. Recruitment was conducted via the following three main mechanisms: (1) web-based recruitment (paid and unpaid), (2) community-based recruitment through partners, and (3) participant referrals via a personalized link. Upon completion of the baseline survey, participants were randomly assigned to one of the following three retention incentive conditions: (1) guaranteed incentive (US $10), (2) lottery incentive (US $50 weekly lottery drawing), and (3) preferred method (guaranteed or lottery). Analyses examined cost per survey start by recruitment source, distribution of demographic characteristics across incentive conditions, and retention by study condition at 3-month and 6-month follow-ups.

Results: Over a 10-week period in 2019, we recruited 480 eligible youth (aged 12-17 years) and 1037 eligible young adults (aged 18-25 years) to the Policy and Communication Evaluation (PACE) Vermont Study. Facebook and Instagram advertising produced the greatest number of survey starts (n=2013), followed by posts to a state-wide web-based neighborhood forum (n=822) and Google advertisements (n=749). Retention was 78.11% (1185/1517) at 3 months and 72.18% (1095/1517) at 6 months. Retention was equivalent across all incentive study conditions at both waves, despite a strong stated preference among study participants for the guaranteed payment at baseline. Youth had greater retention than young adults at both waves (wave 2: 395/480, 82.3% vs 790/1037, 76.18%; wave 3: 366/480, 76.3% vs 729/1037, 70.30%). Substance use prevalence in this cohort was similar to national and state-level surveillance estimates for young adults, but was lower than state-level surveillance estimates for youth. Most participants retained at wave 3 provided positive qualitative feedback on their experience.

Conclusions: Our study supports the feasibility of recruiting a web-based cohort of youth and young adults with representation across an entire state to evaluate substance use prevention efforts. Findings suggest that a guaranteed payment immediately upon survey completion coupled with a bonus for completing all survey waves and weekly survey reminders may facilitate retention in a cohort of youth and young adults.