Publication

Aug 01, 2015

Overview

201532-pic1Problem behaviors in early childhood (birth to five) are associated with poor outcomes in adolescence and adulthood, including delinquency, engagement in criminal activity and violence, and depression (Bornstein, Hahn, & Haynes, 2010; Liu, 2004; O’Connell, Boat, & Warner, 2009). These problem behaviors can be externalizing or internalizing. Externalizing behaviors are directed outward and include aggression, disruptive behavior, and oppositional defiance; internalizing behaviors are directed inward and include withdrawal, anxiety, or depression.

This research brief synthesizes experimental evaluations of 50 programs. The evaluations assessed program impacts on externalizing behaviors and/or internalizing behaviors among children ages birth to five. Evaluations of 27 programs assessed externalizing behaviors exclusively; 22 program evaluations assessed both externalizing and internalizing behaviors; and one assessed internalizing behaviors exclusively. Most of the evaluations focused on preschool children, or those ages three to five.

Key Findings

Overall, 35 of the 50 programs were found to have positive (that is, beneficial) and
statistically significant (p<.05) impacts on either externalizing or internalizing
behaviors.

  • A slightly higher proportion of positive significant impacts were found for externalizing behaviors (63 percent of programs worked) than for internalizing behaviors (52 percent of programs worked).

Positive, significant impacts on externalizing behaviors were more frequent when
programs:

  • provided specific training to parents or teachers, or delivered programming using multiple approaches;
  • were based in preschools;
  • used trained facilitators or therapists to deliver the programs; and/or
  • had a standardized curriculum.

Positive significant impacts on internalizing behaviors were more frequent when programs:

  • targeted children versus parents or teachers;
  • were based in preschools;
  • used teachers to deliver the programs; and/or
  • specifically targeted internalizing behaviors.

Although eight of 22 programs successfully reduced both internalizing and externalizing behaviors, no clear commonalities emerged across their implementation approaches.

Background

The need for effective interventions that address these problem behaviors early is evident from the research. Young children who exhibit these behaviors may experience more difficulty in bonding to family members and forming healthy peer relationships, though cause and effect can be difficult to disentangle (Morris, Silk, Steinberg, Myers, & Robinson, 2007; Shonkoff & Phillips, 2000). Furthermore, these behaviors can interfere with academic performance (Masten et al., 2005). Recent research finds that a child’s level of social competence in kindergarten, which includes their ability to control behaviors and regulate emotions, can predict life outcomes at age 25, including high school graduation, employment, and arrests (Jones, Greenberg, & Crowley, 2015). Intervention during early childhood can prevent behaviors from becoming routine or more serious as the child grows older (Powell, Dunlap, & Fox, 2006). Therefore, it is critical to identify the best approaches for reducing these problem behaviors in early childhood and to highlight the areas where more research is needed.

About the Study

This brief synthesizes findings from random-assignment, intent-to-treat evaluations;a of 50b programs for children ages birth to five, that assessed impacts on either internalizing and/or externalizing behaviors. Most evaluations included preschool-aged children, ages three through five. All evaluations were retrieved from the Child Trends’ What Works/LINKS database of social interventions designed for children and youth. (LINKS refers to Lifecourse Interventions to Nurture Kids Successfully.)

While social-emotional development encompasses a variety of skills and behaviors, this brief focuses on two specific types of problem behaviors: (a) externalizing behaviors, and (b) internalizing problems. For this synthesis, externalizing behaviors are defined as “problems with noncompliance, and antisocial, aggressive, and other uncontrolled behaviors” (Bornstein et al., 2010). Internalizing problems include “withdrawal and anxiety, as well as feelings of inferiority, self-consciousness, shyness, hypersensitivity, and somatic complaints” (Bornstein et al., 2010), although this review did not uncover any evaluations that assessed somatic complaints.c

Findings for the programs in this study are reported using the following categories:d

Found to Work

  • A program evaluation was coded as “found to work” if a positive or beneficial statistically significant impact (p<0.05) was found for the full sample on the majority of relevant outcome measures. An evaluation with multiple follow-ups was coded as “found to work” if a positive statistically significant impact was found at the majority of follow-ups, or at the final follow-up, suggesting a delayed impact. Programs with multiple evaluations were coded as “found to work” if a pattern of positive significant impacts was found in the majority of evaluation.

Mixed Findings

  • A program evaluation was coded as having “mixed findings” if it found significant subgroup differences in effectiveness (for example, by gender or race/ethnicity). Also, an evaluation with multiple follow-ups was coded as having “mixed findings” if impacts on a particular outcome were split, such that half showed positive significant impacts and half had mixed, null, or negative findings. Programs with multiple evaluations were coded as having “mixed findings” if the evaluations were split evenly between not working and working for a particular outcome.

Not Found to Work

  • A program evaluation was coded as “not found to work” if positive significant impacts were found for fewer than half of the relevant outcome measures. An evaluation with multiple follow-ups was coded as “not found to work” if positive significant impacts were found in fewer than half of the followups. Programs with multiple evaluations were coded as “not found to work” if fewer than half of the evaluations found a pattern of positive significant impacts.

These categories were applied when at least four programs were identified that shared a particular characteristic, such as the setting of the intervention, or used a specific approach in common, such as home visiting or parent training. However, no formal meta-analysis was conducted, so these findings should be considered preliminary. Where a given approach was used by three or fewer programs and the approach was supported by existing research or practice wisdom, we discuss the approach in the section on “Needed research.”

Findings

Overall, 35 of the 50 rigorously evaluated programs were found to have a positive impact on either externalizing or internalizing behaviors.e

  • 31 of 49 programs (63 percent) improved externalizing behaviors.
  • 12 of 23 programs (52 percent) improved internalizing behaviors.
  • Of the 22 programs that assessed impacts on both behaviors, eight (36 percent) worked for both
    internalizing and externalizing behaviors.

Overall, interventions characterized by a variety of approaches, settings, targets, and providers worked to reduce externalizing behaviors, suggesting that this cluster of behaviors can be improved using a number of different approaches.

Fewer evaluations focused on interventions to reduce internalizing behaviors, which reflects the pattern of research in the field of socio-emotional development more generally. Internalizing behaviors are less frequently assessed among young children, in part because they can be difficult to measure when a child’s verbal skills are limited (Tandon, Cardeli, & Luby, 2009). However, 24 programs were identified that assessed impacts in this area.

Almost all programs included in this synthesis targeted children who were considered at risk of negative life outcomes, such as substance abuse or arrest, because of their behavior and/or because they were from low-income households. Few programs targeted children with diagnosed disorders, such as oppositional defiant disorder or post-traumatic stress disorder, and most programs were intentionally focused, at least in part, on improving either externalizing or internalizing behaviors.

EXTERNALIZING OUTCOMES

Thirty-one programs demonstrated positive (that is, beneficial) and statistically significant impacts on externalizing behaviors among young children birth to five.f Thirteen programs were found not to work, and five programs had mixed impacts on externalizing behaviors.

Found to Work

Variety of Intervention approaches

Several intervention approaches were found to significantly improve or reduce the frequency of externalizing behaviors. Programs that provided training for parents were the approach most commonly evaluated, and these were successful overall: 13 of 161 were found to work. Teacher trainingg also worked in four of six evaluations2, as did child-targeted interventions, with five of eight found to work.

Variety of program settings

Programs delivered in variety of settings were found to work. Most programs took place in preschools; 11 of 20 of these worked.4 Seven of 12 home-based programs worked;5 six of eight programs based in a community space worked;6 four of five programs taking place in a therapist’s office worked;7 and three of four programs that took place in multiple settings during their implementation, worked.8

Program providers

Most programs were delivered by a trained facilitator;h 22 of 35 programs implemented by a trained facilitator worked.9 Programs using therapists (four of six10) as the program provider also worked, as did programs that used more than one type of provider (four of six11).

Curriculum

Twenty-five programs used a specified curriculum; of these, 16 worked for improving externalizing behaviors.12

Mixed Findings

Home visiting programsi had mixed findings for reducing externalizing behaviors. Four of eight such programs were found to work.13 All home-visiting programs targeted parents only.

Program providers

Programs using teachers (four of eight14) as the program provider had mixed findings for reducing externalizing behaviors.

Not Found to Work

Programs using nurses or doctors as the primary facilitators were, in general, not found to work for this outcome; only one of four worked for reducing externalizing behaviors.15 All of these programs targeted parents in their intervention. Eleven interventions used more than one approach, including a mixture of child, parent, and/or teacher training, along with a combination of case management, therapy or home visiting; five of these 11 interventions worked.16

INTERNALIZING OUTCOMES

Of the 23 programs that assessed internalizing behaviors, 13 demonstrated positive (that is, beneficial) and statistically significant impacts on internalizing behaviors among young children. Seven programs were not found to work; four had mixed results. Fourteen of 23 programs (61 percent) that assessed internalizing behaviors explicitly targeted the reduction of these behaviors as a goal.

Found to Work

Intervention type

Four of four programs that were child-focusedj worked.17

Provider

Four of five programs where teachers delivered the program worked for reducing internalizing behaviors.18

Setting

Interventions (seven of 10) based in preschools worked for reducing internalizing behaviors.19

Targeting Internalizing

Of the 14 programs that explicitly focused on reducing internalizing behaviors, eight worked to reduce these behaviors.20

Mixed Findings

Interventions that were conducted exclusively as home visiting or as parent-training had mixed impacts: for each type, two of four worked.21 Similarly, half (two of four) of interventions that had a nurse or doctor as the primary facilitator worked.22 Providing training to any type of facilitator saw mixed results for internalizing behaviors (nine of 18 that provided training achieved reduced internalizing behaviors).23 Similarly, having a specific curriculum produced mixed results for reducing internalizing behaviors (four of nine such programs worked).24 It is important to note, however, that these findings do not suggest that training and curriculum are not a beneficial practice; the mixed results of this practice could be due to many other factors and is may not be directly tied to training or use of curriculum. Alternatively, it may be that only some types of training or curricula are effective.

Not Found to Work

Interventions that used more than one approach (e.g., parent-training and teacher-training) were not found to work: only two of eight of these worked for reducing internalizing behaviors.25 Targeting both children and parents did not work: only one of four such programs had positive impacts.26

Interventions that used trained facilitators (a facilitator other than teachers, therapists, etc.) were not found to work (four of nine had positive impacts27), nor were interventions with more than one type of facilitator (only one of four such interventions worked).28

BOTH INTERNALIZING AND EXTERNALIZING BEHAVIORS

Of the 23 that assessed impacts on both, eight programs worked for reducing both internalizing and externalizing behaviors.

Found to Work

No specific intervention approaches characterized the majority of successful programs.

Mixed Findings

Parent trainingk and child-focused interventions had mixed findings: for each type, two of four programs worked for both internalizing and externalizing behaviors.29 Programs using teachers as providers had mixed findings (two of five worked for both behaviors).30 Programs based in a school had mixed results (four of nine worked).31

Not Found to Work

Reducing both internalizing and externalizing behaviors with a single program appears to be difficult. Just one of four home visiting programs32 and one of eight programs using more than one type of intervention33 worked for reducing both types of behaviors. Using trained facilitators was not found to work for reducing both behaviors (two of eight worked for both);34 neither was using nurses or doctors as providers (just one of four worked),35 nor using more than one type of provider (one of four worked).36 Also, providing training, regardless of provider type, did not work (six of 18 such programs worked for improving both types of behaviors).37

Having a specific curriculum was also found not to work in reducing both types of behaviors (two of ten such programs worked).38

TREATMENT VS. UNIVERSAL AND TIERED PROGRAMS

Ten programs reviewed were treatment programs that worked with children, or with families of children, who were already demonstrating internalizing or externalizing behaviors. Seven of these worked for reducing either internalizing or externalizing behaviors. Thirty-eight programs were universal in their approach: they worked at the classroom or community level with all children or parents, regardless of their current behavioral status. Of universal programs, twenty-seven worked for either internalizing or externalizing behaviors. Two programs were tiered, meaning they worked both at a universal level, and more intensively with children or families who were demonstrating, or at risk of demonstrating, these behaviors. One of these tiered programs worked for both internalizing and externalizing, the other only worked for internalizing.

NEEDED RESEARCH

A variety of program approaches have been found to be effective for reducing early childhood externalizing behaviors. However, while programs that focused intentionally on internalizing behaviors among young children were more successful in reducing them than programs that did not, fewer have been found to work for internalizing than for externalizing behavior. More evaluations for programs intended to reduce internalizing behaviors among this age group are needed in order to draw strong conclusions about successful intervention approaches. There is also a need for more widespread use of valid screening and assessment instruments for internalizing behavior among young children, if we are to expand this knowledge base.

Testing innovative program delivery approaches is another important area for future research. Reaching working parents and hard-to-reach, marginalized populations through self-guided training using technology, for example, is a promising approach for developing and scaling up effective programs. A few such promising programs that rely on technology emerged in this review: Triple-P Online, containing eight self-guided, online training modules, worked in reducing externalizing behaviors, as did the Telephone-Assisted Behavioral Family Intervention in Germany, which gave families a parenting book and access to regular telephone calls with a child psychologist. This intervention also reduced internalizing behaviors.

Some programs were able to achieve impacts relatively quickly, and their approaches should be studied further. For example, Family Check-up reduced externalizing behaviors after 7.5 hours of home visiting. Six programs reduced externalizing behaviors after spending 12 hours or less with participants. Delivering efficient and effective programming in short-term interventions is an area deserving further research.

Incredible Years adaptations*

Several adaptations of The Incredible Years program have been evaluated, including multiple intervention approaches. The Incredible Years program is a series of training programs to prevent problem behaviors and promote social, emotional, and academic competence. In findings presented elsewhere in this document, three general types of the Incredible Years training are included as separate programs. These are parent-training only, teacher-training only, and child-focused only. All adaptations of the Incredible Years (including combinations of these three types) worked for reducing externalizing behaviors, except for an adaptation that included home-visiting, which had mixed findings.

*Incredible Years adaptations have not been frequently assessed, so the findings from this program include fewer than four evaluations.

Discussion

Many program approaches are successful at reducing externalizing behaviors among young children. Programs targeting parents and teachers are especially successful and should continue. Innovative approaches to program delivery, including technology-based or self-guided training, should be explored to scale up these successful programs and reach families who struggle to attend trainings or commit to home-visiting. While programs that used teachers as facilitators and focused
on children were more successful at reducing internalizing behaviors, more research is needed on what works best to reduce these behaviors among young children; few approaches were successful across programs.

In addition, longitudinal studies following participants over time are especially important for assessing the longer-term benefits of these programs and of other early intervention efforts.

201532-pic2We’d love to hear your thoughts on this publication. Has it helped you or your organization? Email us at feedback@childtrends.org.

Child Trends thanks the Alexander and Margaret Stuart Trust for their generous support of this research brief.

Copyright 2015 by Child Trends, Inc. Publication #2015-32

Child Trends is a nonprofit, nonpartisan research center that studies children at all stages of development. Our mission is to improve outcomes for children by providing research, data, and analysis to the people and institutions whose decisions and actions affect children. For additional information, including publications available to download, visit our website at childtrends.org.


aEvery study has a treatment group and a control group, and random assignment (that is, a lottery system) was used to determine placement of participants into treatment and control groups. Analysis is conducted based on assignment to the treatment group or the control group, not whether or not the subject(s) received the treatment.

Three programs used different intervention approaches among their evaluations: Incredible Years (3 approaches), Parent-Child Interaction Therapy (2 approaches), and Triple P (3 approaches). For the purposes of understanding the implications of these approaches, these interventions are included in the findings as separate programs, even though they are adaptations or variations of an established program brand or name. More information on these programs is provided in Tables 1 and 2.;

cSomatic complaints refer to chronic complaints of physical ailments. Sometimes these complaints persist despite negative evaluations by medical professionals.

dNote: These coding methods are identical to those used for other Child Trends/LINKS Syntheses.

eTwenty-six programs assessed impacts on externalizing behaviors only; one program assessed impacts on internalizing behaviors only; the remaining 23 programs assessed impacts on both externalizing and internalizing behaviors.

f Most programs assessed impacts on 3- and 4-year-olds.

g In this type of intervention, teachers are trained and/or receive coaching on behavior management and promoting social-emotional skills.

h A staff member without a specialized degree, such as a therapist or teacher

iParents and/or children are trained or coached in the home

j These interventions have a set curriculum that is taught to children, typically in preschools

k Parents or caregivers are trained in group or individual sessions outside of the home.

Table

201532-chartkeyprob behaviors chart

Appendix

Intervention types:

“Child-focused”: These interventions have a set curriculum that is taught to children, typically in
preschools.

“Trained facilitator”: A staff member without a specialized degree, such as a therapist or teacher

“Teacher training”: In this type of intervention, teachers are trained and/or receive coaching on behavior management and promoting social-emotional skills.

“Home visiting”: Parents and/or children are trained or coached in the home.

“Parent training”: Parents or caregivers are trained in group or individual sessions outside of the home.

“More than one type”: An intervention that includes elements of more than one of the approaches above.

References

Bornstein, Hahn, C.-S., & Haynes, O. M. (2010). Social competence, externalizing, and internalizing behavioral adjustment from early childhood through early adolescence: Developmental cascades. Development and psychopathology, 22(04), 717-735.

Jones, D. E., Greenberg, M., & Crowley, M. (2015). Early Social-Emotional Functioning and Public Health: The Relationship Between Kindergarten Social Competence and Future Wellness. American Journal of Public Health, e1-e8. doi: 10.2105/AJPH.2015.302630

Liu, J. (2004). Childhood externalizing behavior: theory and implications. Journal of child and adolescent psychiatric nursing, 17(3), 93-103.

Masten, A. S., Roisman, G. I., Long, J. D., Burt, K. B., Obradović, J., Riley, J. R., . . . Tellegen, A. (2005). Developmental cascades: linking academic achievement and externalizing and internalizing symptoms over 20 years. Developmental psychology, 41(5), 733.

Morris, A. S., Silk, J. S., Steinberg, L., Myers, S. S., & Robinson, L. R. (2007). The role of the family context in the development of emotion regulation. Social development, 16(2), 361-388.

O’Connell, M. E., Boat, T., & Warner, K. E. (2009). Preventing Mental, Emotional, and Behavioral Disorders Among Young People: Progress and Possibilities: National Academies Press.

Powell, D., Dulnap, G., Fox, L. (2006). Prevention and Intervention for the Challenging Behaviors of Toddlers and Preschoolers: Infants and Young Children, 19(1), 25-35.

Shonkoff, J. P., & Phillips, D. A. (2000). From Neurons to Neighborhoods: The Science of Early Childhood Development: The National Academies Press.

Tandon, M., Cardeli, E., & Luby, J. (2009). Internalizing disorders in early childhood: A review of depressive and anxiety disorders. Child and adolescent psychiatric clinics of North America, 18(3), 593-610.

Endnotes

[1] Worked: Even Start 2003, DARE to Be You 1998, Incredible Years 1984, 1988, 1990, 1994, 1998, 2004, 2006 (parent training only), Legacy for Children 2013, Parent Management Training – Oregon Model 1982, Parent-Child Interaction Therapy 2007 (for disruptive behavior), ParentCorps 2011, Telephone-Assisted Behavioral Family Intervention 2013, Triple P 2000, 2002, 2003, 2012 (EBFI, SBFI, SDBFI, and Online), Tuning into Kids 2009 Mixed: Family Foundations 2008, 2010 Did not work: Toddlers without Tears 2008, Parent-Child Interaction Therapy 2003, 2004 (for Oppositional Defiant Disorder)

[2] Worked: Learning Language and Loving It 2004, Chicago School Readiness Project 2009, Incredible Years 2001 (teacher training only), Foundations of Learning 2010 Did not work: PATHS Preschool 2007, 2014 Did not work: Tools of the Mind-Play 2014

[3] Worked: Creative Dance and Movement 2006, Incredible Years (child only), Interpersonal Skills Program 1984, Ready to Learn 1999, Tools of the Mind 2006, 2008,(2014 did not work) Mixed: You Can’t Say You Can’t Play 2003 Did not work: Al’s Pals 2004, Fun FRIENDS 2013

[4] Worked: Creative Dance and Movement 2006, Chicago School Readiness Project 2009, Foundations of Learning 2010, Incredible Years 2001 (teacher training only), Interpersonal Skills Program 1984, Learning Language and Loving It 2004, ParentCorps 2011, Prevention Program for Externalizing Problem Behavior (PEP) 2010, Ready to Learn 1999, Tools of the Mind 2006, 2008, (2014 did not work), Tuning into Kids 2009 Mixed: Head Start REDI 2008, Pre-K RECAP 2005, You Can’t Say You Can’t Play 2003 Did not work: Tools of the Mind-Play

[5] Worked: Family Check-Up 2006, Family Spirit 2013, Home-Based Nurse Intervention 2001, Telephone-Assisted Behavioral Family Intervention 2013, Triple P 2000, 2002, 2012, (EBFI, Online), UCLA Family Development Project 1999, 2001 Did not work: Child Health Supervision 1977, Home-Visiting Intervention for Depressed Mothers and Their Infants 2008, Nurse-Family Partnership 2002 2004a 2004b, Promoting First Relationships 2012, Video-feedback Intervention to Promote Positive Parenting and Sensitive Discipline 2006

[6] Worked: DARE to Be You 1998, Incredible Years 1997, 2008 (child only), Incredible Years 1984, 1988, 1990, 1994, 1998, 2004, 2006 (parent training only), Kids in Transition to School (KITS) 2012, Legacy for Children 2013, Making Choices and Strong Families 2014 Mixed: Family Foundations 2008, 2010 Did not work: Toddlers without Tears 2008

[7] Worked: Parent Management Training – Oregon Model 1982, Parent-Child Interaction Therapy 2007 (for disruptive behavior), Triple P 2000, 2002 (SBFI, EBFI) Did not work: Parent-Child Interaction Therapy 2003, 2004 (for Oppositional Defiant Disorder)

[8] Worked: Early Head Start 1999, 2001, 2002, 2010, Even Start 2003, First Step to Success 1998 Did not work: Multi-Method Psycho-Educational Intervention 2000

[9] Worked: Chicago School Readiness Project 2009, Creative Dance and Movement 2006, Family Check-Up 2006, Family Spirit 2013, First Step to Success 1998, Foundations of Learning 2010, Home-Based Nurse Intervention 2001, Incredible Years 1997, 2008 (child only), Incredible Years 1984, 1988, 1990, 1994, 1998, 2004, 2006 (parent training only), Incredible Years 2001 (teacher training only), Interpersonal Skills Program 1984, ParentCorps 2011, Parent Management Training – Oregon Model 1982, Making Choices and Strong Families 2014, Prevention Program for Externalizing Problem Behavior (PEP) 2010, Ready to Learn 1999, Tuning into Kids 2009, Triple P 2000, 2002 (EBFI, SBFI, SDBFI), Tools of the Mind 2006, 2008, (2014 did not work), UCLA Family Development Project 1999, 2001 Mixed: Head Start REDI 2008, Pre-K RECAP 2005, You Can’t Say You Can’t Play 2003 Did not work: Al’s Pals 2004, Fun FRIENDS 2013, Head Start 2010, Home-Visiting Intervention for Depressed Mothers and Their Infants 2008, Multi-Method Psycho-Educational Intervention 2000, Nurse-Family Partnership 2002, 2004a, 2004b, PATHS Preschool 2007, 2014 Promoting First Relationships 2012, Toddlers without Tears 2008, Video-feedback Intervention to Promote Positive Parenting and Sensitive Discipline 2006

[10] Worked: Incredible Years 1997, 2008 (child only) Parent-Child Interaction Therapy 2007 (for disruptive behavior), Parent Management Training – Oregon Model 1982, Telephone-Assisted Behavioral Family Intervention 2013 Mixed: Trauma-informed cognitive behavioral therapy for posttraumatic stress disorder 2011 Did not work: Parent-Child Interaction Therapy 2003, 2004 (for Oppositional Defiant Disorder)

[11] Worked: Early Head Start 1999, 2001, 2002, 2010, Even Start 2003, First Step to Success 1998, ParentCorps 2011 Mixed: Head Start REDI 2008 Didn’t work: Multi-Method Psycho-Educational Intervention 2000

[12] Worked: DARE to Be You 1998, First Step to Success 1998, Incredible Years 1997, 2008 (child only), Incredible Years 1984, 1988, 1990, 1994, 1998, 2004, 2006 (parent training only)

Incredible Years 2001 (teacher training only), Learning Language and Loving It 2004, Parent-Child Interaction Therapy 2007 (for disruptive behavior), ParentCorps 2011 Prevention Program for Externalizing Problem Behavior (PEP) 2010, Ready to Learn 1999, Telephone-Assisted Behavioral Family Intervention 2013, Tools of the Mind 2006, 2008, (2014 did not work), Triple P 2000, 2002 (EBFI, SBFI, SDBFI), Tuning into Kids 2009 Mixed: Head Start REDI 2008, You Can’t Say You Can’t Play 2003, Trauma-informed cognitive behavioral therapy for posttraumatic stress disorder 2011 Didn’t Work: Al’s Pals 2004, Nurse-Family Partnership 2002, 2004a, 2004b, Parent-Child Interaction Therapy 2003, 2004 (for Oppositional Defiant Disorder), PATHS Preschool 2007, 2014,  Promoting First Relationships 2012, Toddlers without Tears 2008

[13] Worked: Family Check-Up 2006, Family Spirit 2013, Home-Based Nurse Intervention 2001, UCLA Family Development Project 1999, 2001 Did not work: Child Health Supervision 1977, Home-Visiting Intervention for Depressed Mothers and Their Infants 2008, Promoting First Relationships 2012, Video-feedback Intervention to Promote Positive Parenting and Sensitive Discipline 2006

[14] Worked: Chicago School Readiness Project 2009, Foundations of Learning 2010, , Ready to Learn 1999, Tools of the Mind 2006, 2008, (2014 did not work) Did not work: Al’s Pals 2004, Fun FRIENDS 2013, Head Start 2010, Did not work: Tools of the Mind-Play

[15] Worked: Home-Based Nurse Intervention 2001 Did not work: Child Health Supervision 1977, Nurse-Family Partnership 2002, 2004a, 2004b, Toddlers without Tears 2008

[16] Worked: Early Head Start 1999, 2001, 2002, 2010 First Step to Success 1998, Kids in Transition to School (KITS) 2012, Making Choices and Strong Families 2014, Prevention Program for Externalizing Problem Behavior (PEP) 2010 Mixed: Head Start REDI 2008, Pre-K RECAP 2005, Trauma-informed cognitive behavioral therapy for posttraumatic stress disorder 2011 Did not work: Head Start 2010, Nurse-Family Partnership 2002 2004a 2004b, Multi-Method Psycho-Educational Intervention 2000

[17] Worked: Al’s Pals 2004, Creative Dance and Movement 2006, Fun FRIENDS 2013, , Marvelous Me 1991

[18] Worked: Al’s Pals 2004, Fun FRIENDS 2013, Foundations of Learning 2010, Chicago School Readiness Project 2009 Did not work: Head Start 2010

[19] Worked: Al’s Pals 2004, Chicago School Readiness Project 2009, Creative Dance and Movement 2006

Foundations of Learning 2010, Fun FRIENDS 2013, Marvelous Me 1991, ParentCorps 2011 Mixed: Pre-K RECAP 2005, PATHS Preschool 2007. (2014 did not work) Did not work: Head Start, 2010

[20] Worked: Al’s Pals 2004, Chicago School Readiness 2009, Child Health Supervision 1977, Creative Dance and Movement 2006, Fun FRIENDS 2013, Making Choices and Strong Families 2014, Marvelous Me 1991, Trauma-informed cognitive behavioral therapy for posttraumatic stress disorder 2011 Did not work: Early Head Start, PATHS Preschool 2007, 2014 Pre-K RECAP 2005, Promoting First Relationships 2012

[21]Home-visiting: Worked: Home-Based Nurse Intervention 2001, Child Health Supervision 1977,

Didn’t work: Promoting First Relationships 2012, Home-Visiting Intervention for Depressed Mothers and Their Infants 2008; Parent training: Worked: Telephone-Assisted Behavioral Family Intervention 2013,

ParentCorps 2011, Mixed: Family Foundations Didn’t work: Toddlers without Tears 2008

[22] Worked: Child Health Supervision 1977, Home-Based Nurse Intervention 2001 Didn’t work: Nurse-Family Partnership 2002, 2004a, 2004b, Toddlers without Tears 2008

[23] Worked: Al’s Pals 2004, Chicago School Readiness Project 2009, Creative Dance and Movement 2006,

Foundations of Learning 2010, Fun FRIENDS 2013, Home-Based Nurse Intervention 2001, Making Choices and Strong Families 2014, ParentCorps 2011, Trauma-informed cognitive behavioral therapy for posttraumatic stress disorder 2011 Mixed: First Step to Success 1998, PATHS Preschool 2007, (2014 did not work), Pre-K RECAP 2005 Did not work: Head Start, 2010, Home-Visiting Intervention for Depressed Mothers and Their Infants 2008, Multi-Method Psycho-Educational Intervention 2000, Nurse-Family Partnership 2002, 2004a, 2004b, Promoting First Relationships 2012, Toddlers without Tears 2008

[24] Worked: Al’s Pals 2004, ParentCorps 2011, Telephone-Assisted Behavioral Family Intervention 2013, Trauma-informed cognitive behavioral therapy for posttraumatic stress disorder 2011 Mixed: PATHS Preschool 2007, 2014, First Step to Success 1998 Did not work: Nurse-Family Partnership 2002, 2004a, 2004b, Promoting First Relationships 2012, Toddlers without Tears 2008

[25] Worked: Making Choices and Strong Families 2014, Trauma-informed cognitive behavioral therapy for posttraumatic stress disorder 2011 Mixed: Pre-K RECAP 2005, First Step to Success 1998 Didn’t work: Early Head Start 1999, 2001, 2002, 2010, Head Start 2010, Multi-Method Psycho-Educational Intervention 2000, Nurse-Family Partnership 2002, 2004a, 2004b

[26] Worked: Making Choices and Strong Families 2014 Mixed: Didn’t work: First Step to Success 1998 , Head Start 2010, Multi-Method Psycho-Educational Intervention 2000,

[27] Worked: Creative Dance and Movement 2006, Making Choices and Strong Families 2014, Marvelous Me 1991, Trauma-informed cognitive behavioral therapy for posttraumatic stress disorder 2011 Mixed: Pre-K RECAP 2005, PATHS Preschool 2007, 2014 Didn’t work: Promoting First Relationships 2012, Home-Visiting Intervention for Depressed Mothers and Their Infants 2008

[28] Worked: ParentCorps, 2011 Mixed: First Step to Success 1998 Didn’t work: Early Head Start 1999, 2001, 2002, 2010, Multi-Method Psycho-Educational Intervention 2000

[29] Parent-training: Worked: ParentCorps 2011, Telephone-Assisted Behavioral Family Intervention 2013, Didn’t work: Family Foundations 2008, 2010, Toddlers without Tears 2008

[30] Worked: Chicago School Readiness Project 2009, Foundation of Learning 2010, Didn’t work: Al’s Pals 2004, Head Start 2010, Fun FRIENDS 2013

[31] Worked: Creative Dance and Movement, 2006, Chicago School Readiness Project 2009, Foundation of Learning 2010, ParentCorps, 2011 Didn’t work: Al’s Pals 2004, Head Start 2010, Fun FRIENDS, 2013, PATHS Preschool 2007, 2014 Pre-K RECAP 2007

[32] Worked: Home-Based Nurse Intervention 2001, Didn’t work: Child Health Supervision 1977, Home-Visiting Intervention for Depressed Mothers and Their Infants 2008, Promoting First Relationships 2012

[33] Worked: Making Choices and Strong Families 2014 Didn’t work: Head Start 2010, Early Head Start 1999, 2001, 2002, 2010, First Step to Success 1998, Pre-K RECAP 2007, Nurse-Family Partnership 2002, 2004a, 2004b, Multi-Method Psycho-Educational Intervention 2000, Trauma-informed cognitive behavioral therapy for posttraumatic stress disorder 2011

[34] Worked: Creative Dance and Movement 2006, Making Choices and Strong Families 2014 Did not work: Family Foundations 2008, 2010, Home-Visiting Intervention for Depressed Mothers and Their Infants 2008, PATHS Preschool, 2007, 2014 Pre-K RECAP 2007, Promoting First Relationships 2012, Trauma-informed cognitive behavioral therapy for posttraumatic stress disorder 2011

[35] Worked: Home-Based Nurse Intervention 2001 Didn’t work: Child Health Supervision 1977, Nurse-Family Partnership 2002, 2004a, 2004b, Toddlers without Tears 2008

[36] Worked: ParentCorps 2011 Didn’t work: Early Head Start 1999, 2001, 2002, 2010, First Step to Success 1998, Multi-Method Psycho-Educational Intervention 2000

[37] Worked: Creative Dance and Movement 2006, Chicago School Readiness Project 2009, Foundation of Learning 2010, Home-Based Nurse Intervention 2001, ParentCorps 2011, Making Choices and Strong Families 2014 Didn’t work: Al’s Pals 2004, First Step to Success 1998, Fun FRIENDS 2013, Head Start 2010, Home-Visiting Intervention for Depressed Mothers and Their Infants 2008, Multi-Method Psycho-Educational Intervention 2000, Nurse-Family Partnership 2002, 2004a, 2004b, PATHS Preschool 2007, 2014, Pre-K RECAP 2005, Promoting First Relationships 2012, Toddler without Tears 2008, Trauma-informed cognitive behavioral therapy for posttraumatic stress disorder 2011

[38] Worked: Telephone-Assisted Behavioral Family Intervention 2013, ParentCorps 2011 Did not work: First Step to Success 1998, Nurse-Family Partnership 2002, 2004a, 2004b, PATHS Preschool 2007, 2014, Promoting First Relationships 2012, Toddlers without Tears 2008, Trauma-informed cognitive behavioral therapy for posttraumatic stress disorder 2011

Authors