This brief was updated on June 10, 2019. You can download the original text here.
Since 2016, Child Trends has collaborated with Healthy Teen Network, Ewald & Wasserman Research Consultants, and MetaMedia Training International to evaluate Pulse, an innovative web-based mobile app designed to empower young women to take control of their health by providing on-demand access to comprehensive, medically accurate sexual health information. The Pulse evaluation is funded by the U.S. Department of Health and Human Services’ Office of Adolescent Health under a grant to rigorously evaluate “new or innovative approaches to prevent teen pregnancy.” Child Trends has completed impact analyses of data from the first cohort of participants (2016–2018) and will publicly release these results in the future. In 2018 and 2019, the study team recruited a second cohort of participants and is currently conducting analyses. The study team will then publicly release the subgroup findings and a report on the evaluation’s implementation.
This resource provides a brief introduction to the Pulse intervention and describes Child Trends’ unique approach to evaluating Pulse’s impact on teen pregnancy. For this intervention, Child Trends recruited a large sample of mostly Black and Latinx young women completely online. The study team was able to retain most of these women for the duration of the intervention and post-test survey administration by using the app in conjunction with pre-programmed, automatically delivered text messages.
Rates of unplanned pregnancy are highest among women in their older teens and early twenties,1,2 yet the majority of teen pregnancy prevention programs are designed for younger teens. Only 8 percent of the nearly 500,000 teens served by the Office of Adolescent Health’s national Teen Pregnancy Prevention Program from 2010 to 2014 were age 16 or older.3 And despite recent declines in teen birth rates among racial and ethnic minority populations in the United States, Black and Latinx teens continue to experience birth rates more than twice as high as their white peers,2, 4 highlighting the need for program approaches that target racial and ethnic minority populations. To address these gaps, teen pregnancy prevention programs have increasingly incorporated technology-based components, but more conclusive research is needed on the efficacy of these approaches.
Pulse is a web-based mobile health app designed for Black and Latinx women ages 18–20. Through culturally and age-appropriate content, Pulse provides information on birth control, healthy relationships, sexual health and physiology, pregnancy, and utilization of clinical services to encourage users to choose effective birth control, seek reproductive health services, and ultimately, prevent unplanned pregnancies.
The Pulse intervention app has six interactive sections with information on sexual and reproductive health topics. It contains activities that engage young women with the content, such as videos featuring medical professionals, a clinic locator, and appointment reminders.
Intervention and Comparison App Main Pages and Intervention Content Page
Know Your Body promotes sexual health by explaining basic anatomy and physiology, and telling users how to prevent sexually transmitted infections.
Know Your Options supports sexually active women in assessing and selecting the best birth control method for them. It promotes birth control literacy, and includes information about all birth control methods with an emphasis on long-acting reversible contraception.
Get Personal supports women who are considering initiating sexual activity, helps them identify abusive behaviors in their relationships, and demonstrates negotiation of condom use with a partner.
Take Action empowers women to visit a health clinic by guiding them to a local clinic through a clinic finder, and orienting them on expectations in a clinic visit.
Make A Plan orients women who think they may be pregnant, or are pregnant, and who are currently navigating prenatal care services.
Get Savvy answers women’s additional questions in text and video format and directs them to additional web-based resources and hotlines.
From 2016 to 2018, Child Trends used a randomized controlled design to evaluate the impact of Pulse on two measures of unprotected sex (sex without any birth control method, and sex without a hormonal/LARC method), as well as the program’s impacts on knowledge, attitudes, self-efficacy, and intentions. The intervention lasted six weeks and consisted of unlimited access to the app and receipt of text messages. The project team administered an online survey at the end of the six-week intervention, and 86 percent of the sample completed it.
Evaluation key features:
Example of Pulse ads
Example Text Messages
* The study team also recruited through ads on Google AdWords and Snapchat but only recruited two participants through these platforms.
At enrollment, participants had to meet the following criteria:
Recruitment through social media ads resulted in 19,706 unique landings on the Pulse study website.* From those unique landings, the screener was completed 10,660 times (representing 54% of all those who landed on the screener page); 4,790 completions did not meet the study eligibility requirements. Of the 5,870 screener responses accounts that were eligible (representing 30% of the original landings), 4,261 (22%) consented to the study, 3,073 (16%) enrolled in the study, and 2,792 (14%) completed the baseline survey and were randomized (1,399 to intervention and 1,393 to comparison). We then removed 475 scammer or duplicate accounts from the randomized sample (233 from the intervention group and 242 from the control group), resulting in a final randomized sample of 2,317 participants in the evaluation study (representing 12% of those who landed on the screener page). From the evaluation sample, 2,130 participants registered with the app (representing 11% of those who landed on the screener page), and 1,935 (10%) logged into the app. Additionally, 1,851 (9%) participants clicked on at least one of the six Pulse sections and 974 (5% of the original sample landing on the screener page) were “fully engaged” with the app, meaning they clicked on at least one activity within each of the six sections.**
* Analytic data for November 2016 through January 2017 are not available and therefore not included in these estimates.
** The study team agreed that this threshold represented the minimum dosage needed to receive the key elements of the program’s content.
Participants were recruited through ads on Facebook, Instagram, and Twitter. Once participants clicked on an ad, they were taken to a screener. Participants completed the screener and, if they met eligibility criteria, were directed to an online enrollment form, consent form, and baseline survey. Upon completion of the baseline survey, participants were randomized to either the intervention or comparison group.
The evaluation team enrolled 2,317 participants online, most of whom were recruited through Instagram ads. The participants’ characteristics at the time of study enrollment were similar for the intervention and comparison groups.
Characteristics of the intervention sample:
 “Hormonal/long-acting reversible contraceptive (LARC) methods” include the following methods: birth control pills, the shot (for example, Depo Provera), the patch (for example, Ortho Evra), the ring (for example, NuvaRing), IUD (for example, Mirena, Skyla, or Paragard), and implants (for example, Implanon or Nexplanon).
Participant app usage
Although participants were asked at enrollment to view the app, not all participants did. Since Pulse was an online evaluation, we relied on participants to view the app on their own time. We analyzed app usage data to see whether participants viewed the app, and how often they did so.
Throughout the initial evaluation of the Pulse study, Child Trends identified several challenges and successes:
Contributors: Genevieve Martínez-García, Milagros Garrido, Nicholas Sufrinko, and Emily Miller.
This publication was made possible by Grant Number TP2AH000038 from the Office of Adolescent Health (OAH), U.S. Department of Health and Human Services (HHS). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the OAH or HHS.
© Copyright 2023 ChildTrendsPrivacy StatementNewsletter SignupLinkedInYouTube