Program

Oct 17, 2016

OVERVIEW                                                    

The Building Educated Leaders for Life (BELL) Summer Learning Program is an enrichment program that primarily focuses on reducing summer learning loss among children of color from low-income backgrounds. Supplementary aims include improving academic self-concept, parental involvement, and social behavior. In an experimental evaluation of the program’s impact on elementary school students, researchers found that reading improved among enrollees. Improvements in reading time and parental involvement were also found. No significant impact on academic self-concept was found.  In a follow-up experimental evaluation of the program, this time for middle school students, researchers found that students’ math scores improved, but only marginally, and their reading skills showed no improvement relative to the comparison group.

DESCRIPTION OF PROGRAM

Target population: Low-performing elementary and middle school students living in low-income urban communities

Reflecting the philosophy of the BELL program to serve children holistically, improving academic skills and academic self-concept are among the major goals of the program. For the on-site component of the program, BELL utilizes nationally recognized literacy (Summer Success: Reading) and mathematics (Summer Success: Math) curricula that align with state and national mandates. Instruction is provided by experienced teachers and teaching assistants to groups of 15 students, from grades one through seven.  In the elementary school program, each class is held for eight hours per day, five days per week, for a six-week period. Children receive eight hours of literacy instruction, four hours of math instruction, and 6.5 hours of community time each week. Parents are encouraged to read with their children, maintain reading logs, and attend program events. To address non-academic-related issues, every week students attend a speaker series where they dialogue with distinguished community residents. In addition, students engage in enrichment activities that focus on both academic and social skills, teamwork, and leadership.  Finally, all participants are required to participate in at least one community service project over the summer. In the middle school program, each class is held for 6.5 hours per day, five days for week, for five weeks.  The morning is spent on academic learning, and the afternoon focuses on science, physical education, and creative activities.  On Fridays, students can participate in a guest lecture series and/or field trips.

EVALUATION(S) OF PROGRAM

Chaplin, D., & Capizzano, J. (2006). Impacts of a summer learning program: A random assignment study of Building Educated Leaders for Life (BELL). Washington, DC: The Urban Institute/Mathematica Policy Research.

Evaluated population: All children entering grades 1 through 7 in Boston and New York were eligible to enter the program, though recruiting was concentrated on low-income minority children who were not in special education. A total of 1,087 children participated and were randomly assigned to either a treatment group or a control group; however, complete test score data were collected on 835 students. The treatment group was composed of 45 percent black children, 42 percent Hispanic children, and 7 percent other children. The control group was composed of 51 percent black children, 38 percent Hispanic children, and 6 percent other children. The treatment group was 51 percent male, and the control group was 48 percent male.

Approach: The research staff used random assignment to determine which students were accepted into the program. The chance of selection into the program varied, depending on the pool that an applicant was put in for random assignment. The pools varied by grade level, program site, and when the family applied for the program. Towards the end of the program, data were collected from both treatment and control groups, using four sources: program administrative data, parent surveys, student surveys, and student tests. The BELL program provided demographic data obtained from participants’ applications and staff personnel. Information regarding child behaviors, learning activities engaged in by children, parental involvement, and background were elicited through parent surveys. Student surveys focused on students’ perceptions of their academic skills. Student tests measured reading skills (e.g., vocabulary and reading comprehension) only, because learning loss among low-income students is greater for literacy than mathematics.

Results: Students in the treatment group read more books than those in the control group. Students in the treatment group also spent more time in academic activities than those in the control group. Children in the treatment group spent less time doing non-academic activities, such as watching TV and being on the computer, than those in the control group. The only difference in parental activities was that parents in the treatment group were more likely to take computer classes than parents of children in the control group.

There were no differences in children’s academic self concept between the treatment group and the control group. The only differences in parental involvement between the two groups were that parents in the treatment group more frequently encouraged their children to read and read to their children, than those in the control group.

Reading test scores between the two groups did not differ. However, when controlling for the number of school days (children in the control group received 16 days of schooling, and treatment children had received 14 days of the BELL program), the researchers found that program students had significantly higher reading scores than those in the control group.

Somers, M-A, Welbeck, R., Grossman, Jean B., & Gooden, S. (2015). An analysis of the effects of an academic summer program for middle school students. New York, NY: MDRC.

Evaluated population: There were 1,032 students recruited into the study, but complete data (test scores and survey) only from 919.  These included 585 who were in the BELL group, and 334 who were in the comparison non-BELL group. They were recruited from three different school districts – one in the west and two in the southeast – that were implementing the BELL program as a voluntary summer program and were over-enrolled. The treatment and control groups had similar distributions across grades, with 20 percent of students entering sixth grade, 42 percent entering seventh grade, and 39 percent entering eighth grade.  The two groups varied somewhat in their racial/ethnic composition.  Approximately 34 percent of both groups were Hispanic, 44 percent of the BELL group and 45 percent of the non-BELL group were black, 6 percent of the BELL group and 5 percent of the non-BELL group were white, 9 percent of both groups were Asian, and 7 percent of the BELL group and 6 percent of the non-BELL group were another racial category.  In the BELL group, 43 percent of the students were female, and in the non-BELL group 46 percent were.  Approximately 90 percent of both groups were eligible for free/reduced-price lunch.  The BELL group had slightly fewer English-as-a-second-language students, with eight percent relative to 11 percent in the non-BELL group. Eighteen percent and 20 percent of the students in the BELL and non-BELL groups, respectively, had an individualized education program. Approximately 40 percent of BELL students and 37 percent of non-BELL students were proficient in reading, while approximately 42 percent of BELL students and 41 percent on non-BELL students were proficient in math, on the state tests given during the previous spring semester. None of these baseline differences were statistically significant.

Approach: Similar to the first evaluation, in order to measure the impact of the BELL Summer Learning Program, the research staff used random assignment (a lottery) at the student level to determine which students were accepted into the program. Data were collected from the students in three ways: Math and reading assessments were conducted  using two different validated tools: the Group Reading Assessment and Diagnostic Examination (GRADE), and the Group Mathematics Assessment and Diagnostic Examination (GMADE), that were administered in the fall of 2012. Additionally, in the fall of 2012, students were asked to complete a survey about their school engagement.

Results: At the end of the program, no statistically significant differences were found between the BELL group and the non-BELL group in terms of reading, math, or school engagement.  However, a difference on one of the sub-tests in the math test was marginally significant (at the 10-percent level), and the effect-size on the math test was 0.12 (p-value=.094), which is a considerable gain over the summer (the authors note that an effect size of 0.07 is “equivalent to a little over one month of additional learning and is the amount by which students are expected to grow during a five-week period during the regular school year.”)  This suggests that preliminary results point toward more effects in math than in either reading or engagement, where the effect-sizes were near 0 for both.  The authors of the study were careful to point out that, because recruitment was a challenge for this study, the effect-size required for a significant finding was very large, and much greater than might be feasible in a month-long summer program.

SOURCES FOR MORE INFORMATION

References

Chaplin, D., & Capizzano, J. (2006). Impacts of a summer learning program: A random assignment study of Building Educated Leaders for Life (BELL). Washington, DC: The Urban Institute/Mathematica Policy Research.

Somers, M-A, Welbeck, R., Grossman, Jean B., & Gooden, S. (2015). An analysis of the effects of an academic summer program for middle school students. New York, NY: MDRC.

Harvard Family Research Project. (2007). A profile of the evaluation of the BELL Accelerated Learning Summer Program. Retrieved August 9, 2007 from http://hfrp.org/out-of-school-time/ost-database-bibliography/database/bell-accelerated-learning-summer-program

Website: http://www.bellnational.org

KEYWORDS:  Summer Program, Children, Adolescents (12-17), Academic Motivation/Self-Concept/Expectations/Engagement, Elementary School, Urban, Academic Achievement, Service Learning, School-Based, Black/African American, Hispanic/Latino, Males and Females (co-ed), Parent or Family Component, Reading/Literacy, Other Education.

Program information last updated 10/17/2016.

Subscribe to Child Trends

Short weekly updates of recent research on children and youth.