Publication

Dec 01, 2015

Background

emcfpicSocial service programs are typically funded by agencies with a specific mission. Accordingly, evaluations also tend to have a narrow focus, be it drug use, crime, or teen pregnancy. However, research and practitioners’ experience indicate that varied problem behaviors often share root causes. This suggests that effective interventions may actually influence multiple outcomes, whether or not they are designed to do so.

Furthermore, developmental research has identified varied outcome domains (such as health, academic success, etc.) and finds that well-being in one domain often affects well-being in other domains. For example, health affects academic success and academic success affects substance use. So, it seems that good outcomes in one domain may lead to a good outcome in another domain.

The strongest intervention strategy, then, may be an approach that affects multiple outcomes. Such a program would not only be good for children and youth, but it would be economically efficient because, all else equal, a single program would produce multiple positive outcomes. Theoretically, even if only modest effects were produced, the existence of multiple small effects could accumulate to make a given program more cost-effective. That is, unless new components are added to achieve additional outcomes, program costs will not increase and programs that impact multiple outcomes can achieve a number of benefits “for the price of one.”1

Child Trends searched for such programs, identifying a number that have positively affected multiple outcomes according to rigorous evaluations. While they may be measured for certain primary outcomes thought to be directly affected by a program, many programs have been shown to improve other outcomes thought plausible, based on prior research and theory, as well. In this brief, we highlight examples of programs that have conducted rigorous experimental evaluations and found impacts on multiple outcomes.2

Key Findings/Implications

  • Programs that improve multiple outcomes are efficient and deserve more emphasis.
  • The following programs have impacts on multiple outcomes: Children’s Aid Society-Carrera Program, Communities that Care, Familias Unidas, Guiding Good Choices, Incredible Years Series, Keep Safe, Life Skills Training, Multisystemic Therapy, Nurse-Family Partnership, Positive Action, Promoting School-community-university Partnerships to Enhance Resilience, Raising Healthy Children, Teen Outreach Program, and Treatment Foster Care Oregon.

Programs With Multiple Impacts

Children’s Aid Society-Carrera Program (CAS-Carrera)

Impacts on math and verbal PSAT scores, and pregnancy and sexual initiation for females

CAS-Carrera is an intensive, multi-year after-school program. Students enter the program at age 10 or 11 and are served through high school. CAS-Carrera is based on seven core components: (1) education, including tutoring and individual academic plans, (2) a weekly job club, through which students are taught financial literacy, (3) family life and sexuality education, including weekly sexuality education sessions, (4) mental health services, including weekly sessions with a social worker, (5) free medical and dental care, (6) self-expression through music, dance, drama, and writing workshops, and (7) individual sports.

A three-year randomized controlled trial3 (RCT) of the after-school model (the in-school model is currently being evaluated) found significant impacts on sexual initiation and pregnancy for females three years after enrollment in the program (Philliber et al., 2002). Another RCT found full-sample positive impacts on teen pregnancy (including boys’ involvement in a pregnancy), health habits and life organization (such as getting a medical check-up and having work experience), and academic outcomes, including math and verbal scores on the PSAT (Preliminary SAT, a standardized test used to determine whether students qualify for the National Merit Scholarship Program) (Philliber, Kaye, & Herrling, 2001).

In this evaluation, subgroup analyses found male-only impacts on beginning to use marijuana, and decreases in giving birth for female participants. Health-related impacts were significant for the full sample and for male participants when analyzed separately, while positive impacts on PSAT verbal scores were significant in the full sample and for girls, but not boys (impacts on PSAT math scores were significant for the full sample only).

CAS-Carrera is run by the Children’s Aid Society, based in New York City, with a National Accreditation and Training Center to assist with replication in new sites. Agencies implementing the CAS-Carrera program must hire a full-time program coordinator, a full-time community organizer, and part-time coordinators (at a minimum) to lead each of the component areas. Providers participate in some outcomes-monitoring activities, while the developer monitors fidelity to the program model. The CAS-Carrera after-school program is being implemented in 11 states and Washington, DC.

Communities that Care (CTC)

Impacts on adolescent delinquency and substance use

Communities that Care is a community-based program that provides training and materials to community stakeholders and decision makers, to help them select and implement evidence-based programs that address the specific needs of their communities. The program’s ultimate goal is to reduce risk factors and enhance protective factors that predict early initiation of substance use and other risky behaviors, including delinquency.

Findings from a random assignment evaluation of CTC indicate that substance use initiation (alcohol, cigarette, and smokeless tobacco) and delinquent behavior were significantly lower in communities implementing CTC compared with control communities (Hawkins et al., 2008; Hawkins et al., 2009). Quasi-experimental findings from a large (100-community) replication provide further support for the model’s effectiveness, particularly with regard to delinquency (Feinberg et al., 2010).

The Center for Communities that Care is based at the Social Development Research Group at the University of Washington, and is responsible for disseminating the CTC model and supporting communities in all phases of preparation and implementation. The model supports local adoption of appropriate interventions, facilitating broad dissemination. A recent cost-benefit analysis revealed a return of $5.30 for every $1.00 invested in CTC (Kuklinski et al., 2012).

Familias Unidas

Impacts on substance use, sexual risk behaviors, and behavior problems

fuThe Familias Unidas program is a family-based, parent-focused intervention for Hispanic adolescents and their families. Parents engage in a series of parent group meetings, in which they learn skills to protect their adolescents from the risks of substance abuse, conduct problems, and risky sexual behaviors through improvements in family functioning. In these groups, parents learn how to effectively communicate with and monitor their adolescent within a context of love and support. Parents have an opportunity to practice learned skills through family sessions with the adolescent and guidance from the facilitator.

Familias Unidas has been evaluated numerous times through RCTs, which have found impacts on adolescent problem behaviors, sexual risk behaviors, drug use, and family functioning. One RCT found, at the 12-month follow-up, significant positive differences in parental engagement and adolescent behavior problems over time for the experimental group (Pantin et al., 2003). Several evaluations have specifically examined the impact of Familias Unidas on adolescent sexual risk behaviors, and have found positive impacts on outcomes including increased condom use, decreased number of partners, and decreased frequency of sex after consuming drugs or alcohol (Pantin et al., 2009; Prado et al., 2012).

Familias Unidas is based out of the Miller School of Medicine at the University of Miami and has been delivered to about 2,500 families to date. Implementation materials, training, and technical assistance for Familias Unidas are bundled in the program package, which costs $50,000 for each group of 10 facilitators. There is extensive real-time monitoring of implementation by staff from the central Familias Unidas, including weekly review of program session tapes, pre- and post-test data from every participant, and updates of other implementation information, such as attendance.

Guiding Good Choices

Impacts on substance use, depressive symptoms, parental communication, and parent-child relationship quality

Guiding Good Choices, formerly known as Preparing for the Drug Free Years, is a family-focused parenting program intended to reduce the likelihood that children will use drugs or alcohol. Each of the program’s five sessions works to build a specific skill, and children participate in the third session, which covers peer resistance and avoidance skills. Guiding Good Choices is typically implemented by schools or community-based organizations.

RCTs evaluating the impacts of Guiding Good Choices have found that children experienced reduced growth in depressive symptoms and multiple substance use (Mason et al. 2007), and reduced growth in alcohol use (Park et al., 2000) several years after program completion.4 The intervention has also been found to have positive impacts on parent-child relationship quality and parental communication (Kosterman et al., 1997; Rueter, Conger, & Ramisetty-Miller, 1999), although the study by Reuter and colleagues (1999) did not find the program to be effective for mother-child relationship quality specifically.

Materials for Guiding Good Choices are distributed by the Channing-Bete company. There is training for facilitators available by request, and there are pre- and post-test measures for performance monitoring and evaluation. However, the standalone, packaged-for-sale distribution of the program means that formal training is not required for implementers before the program is delivered.

Incredible Years Series (IYS)

Impacts on child behavior, social competence, and parenting practices

The Incredible Years Series is a set of curricula designed to enhance social and emotional competence in children ages two through eight who are at risk for or already exhibiting behavior problems or conduct disorders. A basic parent-training curriculum provides instruction on discipline and helping children learn, and can be supplemented with a more advanced parent-training curriculum or one focused on encouraging academic competence. For teachers, there are several full-day training sessions on classroom management, proactive teaching, and other strategies to promote social-emotional competence in their students. Finally, there is a small-group training program for children exhibiting conduct problems. The IYS curricula may be delivered independently or in conjunction with one another.

There is a large research base supporting the effectiveness of IYS. The program has been found to promote prosocial behavior and decrease negative behavior (Webster-Stratton, 1998; Gardner, Burton, & Klimes, 2006), as much as two years after the program was completed (Reid, Webster-Stratton, & Hammond, 2003). Another study found a positive impact on observed child aggression (Brotman et al., 2008). An evaluation of the teacher and child components also found that the program was effective in improving child engagement in classroom activities, but only for children who had poorer-than-average engagement before the intervention began (Webster-Stratton, Reid, & Stoolmiller, 2008). Also, a number of studies have found that IYS improved parenting practices (Webster-Stratton, 1998; Reid, Webster-Stratton, & Hammond, 2003; Gardner, Burton, & Klimes, 2006; Brotman et al., 2008).

Materials for IYS are available for order from the program website. The Incredible Years organization also offers training and technical assistance by telephone or on-site consultation. The purveyor recommends two hours of phone consultation per month in addition to one on-site consultation per year. Facilitators can have videos of their sessions reviewed for certification and as a form of fidelity monitoring.

Keep Safe

Impacts on substance use, internalizing/externalizing behavior, and sexual risk behavior

The Keep Safe program for youth in foster care aims to prevent serious problem behaviors in adolescence by targeting externalizing and internalizing symptoms during the transition to middle school. The intervention involves six group sessions for youth, based on skill-building to support positive behaviors, and six group sessions for caregivers, focused on positive behavioral reinforcements at home, in school, and in the community. Caregivers are also given home practice assignments and offered weekly follow-up sessions for close to a year after the formal end of the program, while youth participate in individual coaching sessions during this time.

Keep Safe has been evaluated via RCT with a sample of 100 girls in foster care. The evaluation found that the program was effective in reducing internalizing and externalizing problems, marijuana and tobacco use, and sexual risk behaviors (Kim & Leve, 2011; Kim et al., 2013; Smith, Leve, & Chamberlain, 2011). The impacts on substance use and sexual behavior were measured two years after the program concluded, while the impacts on internalizing and externalizing symptoms were significant six months after the baseline assessment (after the group session component, but still during the follow-up sessions for caregivers and youth).

Implemksentation support for Keep Safe includes training for facilitators and co-facilitators, an initial planning consultation, and fidelity monitoring prior to (and, to a reduced degree, following) facilitator certification. The first-year costs of Keep Safe are estimated at $202,450 for an implementation including three cohorts of 36 caregivers.5 At present, Keep Safe is being implemented in New York City as part of a child welfare reform effort, and as part of an RCT, funded by the National Institutes of Health, in San Diego County (P. Chamberlain, personal communication, July 27, 2015).

Life Skills Training (LST)

Impacts on substance use and delinquency

Life Skills Training is a school-based substance-use prevention program with separate versions for students in elementary or middle school. Each version of the program lasts for three years, incorporating skill teaching, demonstration, feedback, and other strategies. The skills taught in the program include general life skills to enhance personal and social competence, as well as specific skills for substance avoidance, such as resisting peer pressure.

There are several RCTs finding that LST has positive impacts on substance use outcomes such as smoking, problem drinking, and use of other drugs (Botvin et al., 1992; Botvin et al., 1995; Botvin et al., 2001a; Botvin et al., 2001b). A version of the program designed to reduce delinquency was found to be effective in doing so, and also had positive impacts reducing the prevalence of fighting in higher-risk students (Griffin, Botvin, & Nichols, 2006). The elementary school version of the program was also found to have positive impacts on smoking (Botvin et al., 2003).

National Health Promotion Associates (NHPA), an organization founded by the developer of LST, manages dissemination of the program materials, including additional resources such as a list of grant announcements that could be used to fund LST implementation. NHPA provides online, on-site, and open training workshops for interested implementers, although the by-request nature of these trainings means that not all LST facilitators implementing the program have been trained by the program developer.

Multisystemic Therapy (MST)

Impacts on violence, recidivism, and mental health

Multisystemic Therapy is a home- and community-based intervention that aims to improve outcomes for families of high-risk young people, especially those already involved in the juvenile justice system. Originally, the model was developed for chronic juvenile offenders ages 12 to 17; however, there are more than a dozen adaptations being studied currently. MST reaches more than 23,000 youth per year through more than 500 teams in 34 states and 15 countries.6

Rigorous evaluations of MST have revealed significant and long-term positive impacts on recidivism rates, frequency of arrest, and severity of offense (Borduin et al., 1995; Henggeler, Melton, & Smith, 1992). MST has been found to have impacts on the use of marijuana, but not cocaine, four years after program completion (Henggeler et al., 2002). Borduin and colleagues (1995) also found that families assigned to receive MST reported significant increases in family cohesion, adaptability, and supportiveness, and a significant decrease in emotional negativity following conflict.

The program is organized and operates under MST Services, a for-profit company licensed by the Medical University of South Carolina. Agencies implementing MST are supported by MST Services through an initial orientation, training, and ongoing consultation, which are all required for certified MST sites. Further, MST has a well-developed quality assurance/quality improvement system for monitoring model adherence across organizational levels.

Nurse-Family Partnership (NFP)

Impacts on mothers’ subsequent births, children’s arrests, and children’s smoking

Nurse-Family Partnership is a home-visiting-based program that uses three strategies to promote the well-being of low-income, first-time mothers and their children. First, it seeks to improve pregnancy outcomes by helping women alter health-related behaviors such as smoking, alcohol consumption, and drug use. Second, it teaches parents how to provide more responsible and competent care in order to improve child health and development. Finally, it works to improve families’ economic self-sufficiency by helping parents plan for future pregnancies, further their education, and secure employment. Program components are provided through the child’s second birthday and include home visits by trained nurses, referrals to community resources, and supervision of visiting nurses by an NFP nurse supervisor.

NFP has been evaluated in a number of RCTs in different regions of the United States, finding positive impacts on mothers’ repeat pregnancies and time between pregnancies. An evaluation examining long-term impacts for children found a number of positive outcomes. Fifteen years after the program, children of mothers in the NFP group had been arrested less, convicted less, violated probation less, smoked fewer cigarettes per day, consumed alcohol less frequently, and had fewer behavioral problems than children of mothers in the control group.

NFP has been active since 1977, when it was first established as a demonstration program in Elmira, New York. Since then, the program has been expanded to serve between 9,000 to 10,000 families in 43 different states annually. Implementation of NFP is centralized; agencies interested in implementing NFP do so after entering into a contract with the central NFP organization, a process that can only take place after a feasibility check by the NFP National Service Office. While the scale of individual implementations varies, all organizations must be able to serve 100 families when starting up. Training for nurses is delivered by the Nursing Practice team at the National Service Office.

Positive Action (PA)

Impacts on substance use, math, reading, violence, sexual behavior, and mental health

Positive Action is a positive youth development program with yearly curricula for children and adolescents in pre-kindergarten through the end of high school. Although it is most often implemented in schools, PA is appropriate for use in other settings, including community-based organizations and the juvenile justice system. In the school-based implementation of PA, 15-minute grade-specific sessions are taught four days a week. PA activities include scripted lessons, role-plays, and question-and-answer sessions, and facilitapators are encouraged to support positive behaviors by role-modeling and reinforcing them outside of sessions. Parent kits for PA provide weekly activities that match the lessons taught by the facilitator.

Evidence supporting PA’s effectiveness in improving several domains of child well-being comes from multiple RCTs in Hawaii, Chicago, and the southeastern United States. A study examining impacts from all three trials found that PA was effective in reducing decline of positive behaviors over time (Washburn et al., 2011). In Hawaii, the program was found to improve numerous outcomes, including substance use, violent behavior, sexual activity, reading and math test scores, and academic behavior (Beets et al., 2009; Snyder et al., 2010; Snyder et al., 2013). PA schools also demonstrated improvements in school quality (Snyder et al., 2012), as well as suspensions and absenteeism (Snyder et al., 2010). In Chicago, PA was found to improve a number of socioemotional outcomes including substance use, disruptive and violent behavior, bullying, depression, and anxiety (Lewis et al., 2013a; Lewis et al., 2013b; Li et al., 2011). PA schools had lower rates of disciplinary referral and suspension than schools in the control group at the end of the study (Lewis et al., 2013b); for students in PA schools, the program was found to reduce increases in academic disaffection, improve academic motivation, and, for African American boys, improve reading test scores (Bavarian et al., 2013).

PA has been implemented in various settings since the early 1980s, with curricula and program materials distributed by the program developer through Positive Action, Inc. Most agencies interested in implementing the program buy the curricula online and purchase training or support services on an as-needed basis. Schools are the most common venue of implementation for PA, but the program can be implemented in community- or faith-based organizations, after-school programs, or detention centers. Most implementations of PA involve some use of public funds, such as Title I funds or substance-use block grants.

Promoting School-community-university Partnerships to Enhance Resilience (PROSPER)

Impacts on substance use and family functioning

PROSPER is a multi-tiered program-delivery system that works to build partnerships among public schools, university extension systems, and prevention scientists, to support local community teams in implementing universal evidence-based programs. These community teams select evidence-based programs from a list supported by PROSPER; one school-based program (for 7th graders) and one family program (for 6th graders), which must support positive youth development, reduce youth risk behaviors, and make families stronger. School-based programs currently offered include All-Stars, LifeSkills Training, and Lions Quest Skills for Adolescence. Family-based programs include the Strengthening Families Program: For Parents and Youth 10-14, and Guiding Good Choices. A community team at each site determines which programs are implemented.

The PROSPER system has been evaluated in a long-term randomized controlled trial involving 28 communities. Positive impacts were found on conduct problems, several outcomes related to substance use, and family functioning outcomes (Redmond et al., 2009; Spoth et al., 2007; Spoth et al., 2011; Spoth et al., 2013a; Spoth et al., 2013b; Spoth et al., 2015). The programs offered as part of the PROSPER system have been evaluated and found to have impacts in multiple areas; their long-term impacts when delivered through the PROSPER system continue to be evaluated.

The PROSPER delivery system was developed by researchers at Iowa State University’s Partnerships in Prevention Science Institute (PPSI) in collaboration with researchers at Pennsylvania State University’s Prevention Research Center. The National PROSPER Network Organization, which is based out of the PPSI, coordinates the efforts of all PROSPER Partnership States and provides each state with tailored, proactive technical assistance on team functioning, program implementation, evaluation, and sustainability. A key feature of the system is its use of a feedback loop, including program implementation evaluation and monitoring of the partnership process, to identify areas in need of technical assistance and progress toward standardized benchmarks.

Raising Healthy Children (RHC)

Impacts on substance use, academic performance, and social competence

Raising Healthy Children is a positive youth development program for children and youth in elementary through high school, with components involving teachers, parents, and students. Teachers attend workshops and are trained in classroom management and teaching strategies that reinforce cooperative learning and good behavior. Parents participate in group workshops focusing on family management skills and problem solving at home, and coordinators provide additional in-home case management services to families of children with conduct problems. Students may participate in other activities depending on age and need. High-risk elementary school students can participate in a summer camp, while middle school students may receive tutoring services. As students approach driving age, they participate in a family-based “Safe Drivers Wanted” program focused on positive decision-making and curbing impulsive behavior on the road.

A series of studies have evaluated the effectiveness of RHC over the course of one long-term RCT. The first evaluation (Fleming et al., 2001) found impacts on unscheduled school transfers for elementary school students. The next evaluation (Catalano et al., 2003) found positive impacts on teachers’ ratings of social competence and antisocial behavior, as well as teacher and parent reports of students’ academic performance and commitment to school. When students reached grade 10, those in RHC schools used marijuana and alcohol less frequently than those in control group schools, although there were no differences in use versus nonuse (Brown et al., 2005).

RHC is a program of the Social Development Research Group (SDRG) at the University of Washington, which supports implementation of the program. SDRG engages in training of trainers to support sustainability, and offers consultation and technical assistance services. SDRG estimates the cost of staff development as about $950 per teacher in the first and second years of the program, which drops to $500 in the third.7 The parenting workshop leader training costs $4,200 plus trainer expenses for a group of 12 trainees.8

Teen Outreach Program (TOP)

Impacts on school suspension, course failure, and pregnancy

topThe Teen Outreach Program, which currently serves about 30,000 youth nationally,9 involves participants in community service learning and curriculum-guided discussion during the school year. Though TOP seeks to prevent problem behaviors—notably, teen pregnancy and school failure—the program curriculum places very little direct emphasis on these two behaviors.

Two RCT evaluations of TOP assessed the program in a school setting and found significant positive impacts on school suspension, course failure, and pregnancy during the intervention period (Allen et al., 1997; Philliber & Allen, 1992). TOP is currently being implemented, and evaluated, in after-school and community-based settings.

TOP is owned and managed by Wyman, a private non-profit organization, as a replication partner network model of 56 national partners including other non-profits, departments of social services, and departments of health. Partner organizations train direct-service providers to deliver TOP. The curriculum is designed to be flexible and the setting, target population, sequence of lessons, and other factors can be tailored, within the bounds of the program logic model. Replication partners must ensure that the core components are delivered by program providers.

Treatment Foster Care Oregon (TFCO)

Impacts on problem behaviors, delinquency, and pregnancy

Treatment Foster Care Oregon, formerly Multidimensional Treatment Foster Care, is an alternative to residential treatment for high-risk adolescents and youth. The goal of TFCO is to provide youth with skills, resources, supervision, and life structure to reduce delinquency and promote more prosocial and adaptive behaviors. Foster families, case managers, and therapists work intensively with the participating youth and his or her family to prepare for reunification or an alternative permanency solution. The participating foster parents receive TFCO-specific training and work as the primary implementers of the behavior-management component of the program, while biological, adoptive, or other permanent caregivers receive family therapy and host home visits for their child. Youth behavior and foster parents’ stress levels are assessed daily by TFCO staff.

Several RCTs provide evidence for TFCO’s effectiveness in improving outcomes for youth. It has repeatedly been found to reduce delinquency, arrest, and incarceration (Chamberlain & Reid, 1998; Eddy, Whaley, & Chamberlain, 2004). Evaluations have also found positive impacts on reducing problem behaviors (Chamberlain & Reid, 1991; Chamberlain & Reid, 1998), as well as on pregnancy rates in delinquent girls (Kerr, Leve, & Chamberlain, 2009).

On any given day, about 1,200 youth are in TFCO programs, which are currently implemented in numerous states as well as several countries in Europe, and New Zealand. TFC Consultants, a company founded by the developers of TFCO, works as a central authority for the program, and meets with implementers before mandatory program training begins. During the implementation period, sites stay in close contact with TFC Consultants through weekly phone consultations and in-person technical assistance. The first-year costs of a TFCO program have been estimated at $533,500.10 A cost-benefit analysis by the Washington State Institute for Public Policy determined that the criminal justice savings from TFCO may range from $21,836 to $87,622 per participant, and estimated the average cost per participant of the program to be about $2,052 more than placing the youth in a typical group home, a benefit-to-cost ratio between $10.64:1 and $47.30:1 for every dollar spent (Aos et al., 2001).

Conclusion

Youth-serving programs are often described in a way that focuses on one outcome, such as a “dropout prevention program” or a “teen pregnancy prevention program.” While many evidence-based programs may indeed have very focused impacts, the programs highlighted in this brief demonstrate that multiple elements of child well-being can be affected by a single well-designed and carefully implemented intervention. Some are multi-generation programs for parents and children. Some are long-term interventions, and others are shorter-term interventions. For some of these programs, impacting multiple outcomes is not only an intentional goal but also a key part of their theory of change; the Familias Unidas intervention, for example, aims to reduce sexual risk behavior by improving family functioning (Prado et al., 2012). As with all evidence-based programs, programs found to impact multiple areas of well-being should be implemented with fidelity to the tested, effective program model. If implementation is not done with fidelity, a program could be less effective than indicated by research.

As the connections between child well-being outcomes are increasingly recognized—for example, that greater school connectedness predicts lower substance use (Sacks, Moore, Terzian, & Constance, 2014)—the impetus for expanding the body of research on programs that impact multiple outcomes is likely to grow as well.

This report was produced with funding from the Edna McConnell Clark Foundation.

Footnotes and References

1 We have not found hard data on whether there are incremental costs associated with a focus on multiple outcomes (beyond the cost of measuring multiple outcomes). We hypothesize that the incremental costs, ifany, are less than the costs of implementing a new initiative to address the additional outcome(s), but this expectation requires empirical analysis.
2 In addition to having been found to impact multiple outcomes, the programs described in this brief are currently providing services, provide written program materials, and have formal training available, either by request or as a pre-implementation requirement. It should be stressed that the programs included in this brief do not constitute a comprehensive list of the programs found to impact multiple outcomes.
3 Randomized controlled trials (RCTs) randomly assign participants to one of at least two groups: a treatment group and a control group. They are considered the gold standard of evaluation, because successful random assignment limits the aggregate differences between treatment and control groups to one: whether they are assigned to receive the treatment or not. This allows evaluators to isolate program impacts and make causal statements.
4 Rates of depressive symptoms, multiple substance use, and alcohol use tend to increase over time, so reduced growth means less likelihood of these outcomes as children age.
5 Blueprints for Healthy Youth Development. Keep Safe. Retrieved June 2, 2015, from http://www.blueprintsprograms.com/programCosts.php?pid=a79e9a409bded1928e5dad9765d53e7bce91d555.
6 MST Services.Team Locations. Retrieved May 21, 2015, from http://mstservices.com/teams/team-locations.
7 Social Development Research Group (2012). Raising Healthy Children Staff Development Program Costs. Retrieved May 21, 2015, from http://www.sdrg.org/rhcsummary3.asp.
8 Blueprints for Healthy Youth Development. Raising Healthy Children. Retrieved May 21, 2015, from http://www.blueprintsprograms.com/programCosts.php?pid=5e06d22c8893e27d5a7243bd185faa94cc593072.
9 Crime Solutions. Program Profile: Wyman’s Teen Outreach Program ®. Retrieved May 21, 2015, from https://www.crimesolutions.gov/ProgramDetails.aspx?ID=277.
10 Blueprints for Healthy Youth Development. Multidimensional Treatment Foster Care. Retrieved May 26, 2015, from http://www.blueprintsprograms.com/programCosts.php?pid=632667547e7cd3e0466547863e1207a8c0c0c549.

Allen, J. P., Philliber, S., Herrling, S., & Kuperminc, G. P. (1997). Preventing Teen Pregnancy and Academic Failure: Experimental Evaluation of a Developmentally Based Approach. Child Development, 64(4), 729-742.

Aos, S., Phipps, P., Barnoski, R., & Lieb, R. (2001). The Comparative Costs and Benefits of Programs to Reduce Crime. Olympia, WA: Washington State Institute for Public Policy.

Bavarian, N., Lewis, K. M., DuBois, D. L., Acock, A., Vuchinich, S., Silverthorn, N., Snyder, F. J., . . . Flay, B. R. (2013). Using social-emotional and character development to improve academic outcomes: A matched-pair, cluster-randomized controlled trial in low-income, urban schools. Journal of School Health, 83(11), 771-779.

Beets, M. W., Flay, B., Vuchinich, R. A., Snyder, F., Acock, A. C., Li, K., Burns, K., Washburn, I., & Durlak, J. A. (2009). Use of a social and character development program to prevent substance use, violent behaviors, and sexual activity among elementary-school students in Hawaii. American Journal of Public Health, 99(8), 1438-1445.

Borduin, C. M., Mann, B. J., Cone, L. T., Henggeler, S. W., Fucci, B. R., Blaske, D. M., Williams, R.A. (1995). Multisystemic treatment of serious juvenile offenders: Long-term prevention of criminality and violence. Journal of Consulting and Clinical Psychology, 63(4), 569-578.

Botvin, G. J., Dusenbury, L, Baker, E., James-Ortiz, S., Botvin, E., Kerner, J. (1992). Smoking prevention among urban minority youth: Assessing effects on outcome and mediating variables. Health Psychology, 11(5), 290-299.

Botvin, G. J., Baker, E., Dunesbury, L., Botvin, E. M. & Diaz, T. (1995). Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. Journal of the American Medical Association, 273(14), 1106-1112.

Botvin, G. J., Griffin, K. W., Diaz, T., Ifill-Williams, M. (2001a). Drug abuse prevention among minority adolescents: Posttest and one-year follow-up of a school-based preventive intervention. Prevention Science, 2, 1-12.

Botvin, G. J., Griffin, K. W., Diaz, T., Ifill-Williams, M. (2001b). Preventing binge drinking during early adolescence: One- and two-year follow-up of a school-based preventive intervention. Psychology of Addictive Behaviors, 4, 360-365.

Botvin, G. J., Griffin, K. W., Paul, E., & Macaulay, A. P. (2003). Preventing tobacco and alcohol use among elementary school students through Life Skills Training. Journal of Child and Adolescent Substance Abuse, 12(4), 1-17.

Botvin, G. J., Griffin, K. W., & Nichols, T. D. (2006). Preventing youth violence and delinquency through a universal school-based prevention approach. Prevention Science, 7, 403-408.

Brotman, L. M., Gouley, K.K., Huang, K., Rosenfelt, A., O’Neal, C., & Klein, R.G. (2008). Preventive intervention for preschoolers at high risk for antisocial behavior: Long-term effects on child physical aggression and parenting practices. Journal of Child and Adolescent Psychology, 37(2), 386-396.

Brown, E. C., Catalano, R. F., Fleming, C. B., Haggerty, K. P., & Abbott, R. D. (2005). Adolescent substance use outcomes in the Raising Healthy Children Project: A two-part latent growth curve analysis. Journal of Consulting and Clinical Psychology, 73, 699-710.

Catalano, R. F., Mazza, J. J., Harachi, T. W., Abbott, R. D., Haggerty, K. P., & Fleming, C. B. (2003). Raising Healthy Children through enhancing social development in elementary school: Results after 1.5 years. Journal of School Psychology, 41(2), 143-164.

Chamberlain, P., & Reid, J.B. (1991). Using a specialized foster care community treatment model for children and adolescents leaving the state mental hospital. Journal of Community Psychology, 19, 266-276.

Chamberlain, P., & Reid, J. B. (1998). Comparison of two community alternatives to incarceration for chronic juvenile offenders. Journal of Consulting and Clinical Psychology, 66(4), 624-633.

Eddy, J. M., Whaley, R. B., & Chamberlain, P. (2004). The prevention of violent behavior by chronic and serious male juvenile offenders: A 2-year follow-up of a randomized clinical trial. Journal of Emotional and Behavioral Disorders, 12(1), 2-8.

Feinberg, M. W., Jones, D., Greenberg, M. T., Osgood, D. W., & Bontempo, D. (2010). Effects of the Communities That Care model in Pennsylvania on change in adolescent risk and problem behaviors. Prevention Science, 11, 163-171.

Fleming, C. B., Harachi, T. W., Catalano, R. F., Haggerty, K. P., & Abbott, R. D. (2001). Assessing the effects of a school-based intervention on unscheduled school transfers during elementary school. Evaluation Review, 25(6), 655-679.

Gardner, F., Burton, J., & Klimes, I. (2006). Randomised controlled trial of a parenting intervention in the voluntary sector for reducing child conduct problems: Outcomes and mechanisms of change. Journal of Child Psychology and Psychiatry, 47(11), 1123-1132.

Griffin, K. W., Botvin, G.J., & Nichols, T.R. (2004). Long-term follow-up effects of a schoo-lbased drug abuse prevention program on adolescent risky driving. Prevention Science, 5, 207-212.

Hawkins, J. D., Brown, E. C., Oesterle, S., Arthur, M. W., Abbott, R. D., Abbott, R. D., Catalano, R. F. (2008). Early effects of Communities That Care on targeted risks and initiation of delinquent behavior and substance use. Journal of Adolescent Health, 43, 15-22.

Hawkins, J. D., Oesterle, S., Brown, E. C., Arthur, M. W., Abbott, R. D., Fagan, A. A., Catalano, R. F. (2009). Results of a type 2 translational research trial to prevent adolescent drug use and delinquency: A test of Communities That Care. Archives of Pediatric and Adolescent Medicine, 163, 789-798.

Henggeler, S. W., Melton, G.B., & Smith, L.A. (1992). Family preservation using Multisystemic Therapy: An effective alternative to incarcerating serious juvenile offenders. Journal of Consulting and Clinical Psychology, 60(6), 953-961.

Henggeler, S. W., Clingempeel, W. G., Brondino, M. J., & Pickrel, S. G. (2002). Four-year follow-up of Multisystemic Therapy with substance-abusing and substance-dependent juvenile offenders. Journal of the American Academy of Child & Adolescent Psychiatry, 41(7), 868-874.

Kerr, D. C. R., Leve, L. D., & Chamberlain, P. (2009). Pregnancy rates among juvenile justice girls in two randomized controlled trials of multidimensional treatment foster care. Journal of Consulting and Clinical Psychology, 77(3), 588-593.

Kim, H. K., Pears, K. C., Leve, L. D., Chamberlain, P. C., & Smith, D. K. (2013). Intervention effects on health-risking sexual behavior among foster care girls: The role of placement disruption and substance use. Journal of Child and Adolescent Substance Abuse, 22(5), 370-387.

Kim, H. L., L. (2011). Substance use and delinquency among middle school girls in foster care: A three-year follow-up of a randomized controlled trial. Journal of Consulting and Clinical Psychology, 76(6), 740-750.

Kosterman, R., Hawkins, D., Spoth, R., Haggerty, K. & Zhu, K. (1997). Effects of a preventive parent-training intervention on observed family interactions: Proximal outcomes from Preparing for the Drug Free Years. Journal of Community Psychology, 25(4), 337-352.

Kuklinski, M. R., Briney, J. S., Hawkins, J. D., & Catalano, R. F. (2012). Cost-benefit analysis of Communities That Care outcomes at eighth grade. Prevention Science, 13(2), 150-161.

Lewis, K. M., Bavarian, N., Snyder, F. J., Acock, A., Day, J., DuBois, D. L., Ji, P., . . . Flay, B. R. (2012). Direct and mediated effects of a social-emotional and character development program on adolescent substance use. International Journal of Emotional Education, 4(1), 56-78.

Lewis, K. M., DuBois, D. L., Bavarian, N., Acock, A., Silverthorn, N., Day, J., Ji, P., . . . Flay, B. R. (2013a). Effects of Positive Action on the emotional health of urban youth: A clusterrandomized trial. Journal of Adolescent Health, 53(6), 706-711.

Lewis, K. M., Schure, M. B., Bavarian, N., DuBois, D. L., Day, J., Ji, P., Silverthorn, N., . . . Flay, B. R. (2013b). Problem behavior and urban, low-income youth: A randomized controlled trial of Positive Action in Chicago. American Journal of Preventive Medicine, 44(6), 622-630.

Li, K., Washburn, I., DuBois, D. L., Vuchinich, S., Ji, P., Brechling, V., Day, J., Beets, M. W., Acock, A. C., Berbaum, M., Snyder, F., & Flay, B. (2011). Effects of the Positive Action program on problem behaviors in elementary school students: A matched-pair randomized control trial in Chicago. Psychology and Health, 26(2), 187-204.

Mason, W. A., Kosterman, R., Hawkins, J. D., Haggerty, K. P., Spoth, R., & Redmond, C. (2007). Influence of a family-focused substance use preventive intervention on growth in adolescent depressive symptoms. Journal of Research on Adolescence, 17(3), 541-564.

Pantin, H., Coatsworth, J. D., Feaster, D. J., Newman, F. L., Briones, E., Prado, G., Schwartz, S. J., & Szapocznik, J. (2003). Familias Unidas: The efficacy of an intervention to promote parental investment in Hispanic immigrant families. Prevention Science, 4(3), 189-201.

Pantin, H., Prado, G., Lopez, B., Huang, S., Tapia, M. I., Schwartz, S. J., Sabillon, E., Brown, C. H., Branchini, J. (2009). A randomized controlled trial of Familias Unidas for Hispanic adolescents with behavioral problems. Psychosomatic Medicine, 71(9), 987-995.

Park, J., Kosterman, R., Hawkins, J.D., Haggerty, K.P., Duncan, T.E., Duncan, S.C., & Spoth, R. (2000). Effects of the “Preparing for the Drug Free Years” curriculum on growth in alcohol use and risk for alcohol use in early adolescence. Prevention Science, 1(3), 125-138.

Philliber, S., & Allen, J.P. (1992). Life options and community service: Teen Outreach Program. In J. J. C. B. C. Miller, L. Paikoff, & J. L. Peterson (Ed.), Preventing adolescent pregnancy: Model programs and evaluation (pp. 139-155). Newbury Park, CA: Sage Publications.

Philliber, S., Kaye, J., & Herrling, S. (2001). The national evaluation of the Children’s Aid Society Carrera-model program to prevent teen pregnancy. New York, NY: Philliber Research Associates.

Philliber, S., Kaye, J. W., Herrling, S., & West, E. (2002). Preventing pregnancy and improving health care access among teenagers: An evaluation of the Children’s Aid Society-Carrera Program. Perspectives on Sexual and Reproductive Health, 34, 244-251.

Prado, G., Pantin, H., Huang, S., Cordova, D., Tapia, M. I., Velazquez, M.-R., Calfee, M., Malcom, S., Arzon, M., Villamar, J., Jimenez, G. L., Cano, N., Brown, C. H., & Estrada, Y. (2012). Effects of a family intervention in reducing HIV risk among high-risk Hispanic adolescents. Archives of Pediatric and Adolescent Medicine, 166(2), 127-133.

Redmond, C., Spoth, R. L., Shin, C., Schainker, L. M., Greenberg, M. T., & Feinberg, M. (2009). Long-term protective factor outcomes of evidence-based interventions implemented by community teams through a community-university partnership. Journal of Primary Prevention, 30, 513-530.

Reid, M. J., Webster-Stratton, C., & Hammond, M. (2003). Follow-up of children who received the Incredible Years intervention for oppositional defiant disorder: Maintenance and prediction of 2-year outcome. Behavior Therapy, 34(4), 471-491.

Sacks, V. H., Moore, K. A., Terzian, M., & Constance, N. (2014). School policies, school connection, and adolescents: What predicts young adults substance use? Publication number 2014-53. Bethesda, MD: Child Trends.

Smith, D., Leve, L., & Chamberlain, P. (2011). Preventing internalizing and externalizing problems in girls in foster care as they enter middle school: Immediate impact of an intervention. Prevention Science, 12(3), 269-277.

Snyder, F. J., Acock, A., Vuchinich, S., Beets, M. W., Washburn, I. J., Flay, B. R. (2013). Preventing negative behaviors among elementary-school students through enhancing students’ social-emotional and character development. American Journal of Health Promotion, 28(1), 50-58.

Snyder, F. J., Vuchinich, S., Acock, A., Washburn, I. J., Flay, B. R. (2012). Improving elementary school quality through the use of a social-emotional and character development program: A matched-pair, cluster-randomized, controlled trial in Hawai’i. Journal of School Health, 82(1), 11-20.

Snyder, F., Vuchinich, S., Acock, A. C., Washburn, I., Beets, M., & Li, K. (2010). Impact of a social-emotional and character development program on a school-level indicators of academic achievement, absenteeism, and disciplinary outcomes: A matched-pair, clusterrandomized, controlled trial. Journal of Research on Educational Effectiveness, 3(1), 26-55.

Spoth, R. L., Trudeau, L. S., Redmond, C., Shin, C., Greenberg, M. T., Feinberg, M. E., & Hyun, G. H. (2015). PROSPER partnership delivery system: Effects on adolescent conduct problem behavior outcomes through 6.5 years past baseline. Journal of Adolescence, 45, 44-55.

Spoth, R., Redmond, C., Shin, C., Greenberg, M., Clair, S., & Feinberg, M. (2007). Substance use outcomes at 18 months past baseline: The PROSPER community-university partnership trial. American Journal of Preventive Medicine, 32(5), 395-402.

Spoth, R., Trudeau, L., Shin, C., Ralston, E., Redmond, C., Greenberg, M., Feinberg, M. (2013). Longitudinal effects of universal preventive intervention on prescription drug misuse: Three RCTs with late adolescents and young adults. American Journal of Public Health, 103(4), 665-672.

Washburn, I. J., Acock, A., Vuchinich, S., Snyder, F., Li, K., Ji, P., Day, J., . . . Flay, B. R. (2011). Effects of a social-emotional and character development program on the trajectory of behaviors associated with social-emotional and character development: Findings from three randomized trials. Prevention Science, 12, 314-323.

Webster-Stratton, C. (1998). Preventing conduct problems in Head Start children: Strengthening parenting competencies. Journal of Consulting and Clinical Psychology, 66(5), 715-730.

Webster-Stratton, C., Reid, M.J., & Stoolmiller, M. (2008). Preventing conduct problems and improving school readiness: Evaluation of the Incredible Years Teacher and Child Training Programs in high-risk schools. Journal of Child Psychology and Psychiatry, 49(5), 471-488.

Authors