Child Trends identified evaluated after-school programs in Boston that: were 501 c3 organizations; served over 50 children; served children and adolescents between the ages of 8 and 16; had an operating budget of $750,000 and higher (with the exception of Camp Fire for Eastern Massachusetts Council and Youth Enrichment Services); and were evaluated. Programs were excluded if they were large institutions such as schools, juvenile detention centers, residential group homes, religious organizations and hospitals.

To compile a list of programs that might meet these criteria, a standardized process was used:

GuideStar Scan: *GuideStar, (the national database of non-profit organizations filing 990s) was used to identify relevant programs based on the criteria specified by Atlantic Philanthropies. A standard set of search terms were used to allow for consistent searching. The search terms used were: children, youth, youth program(s), and child programs. *GuideStar was also searched under the category of “youth development”. Once the relevant program information was identified, it was included in the description of the program.

After-School Program Scan: If information in Guide Star indicated that the program had been evaluated, then the program web site was scanned for relevant evaluation information.

Phone interviews with Community Foundations and Experts: Where individual program web sites did not yield detailed evaluation information, phone interviews with program sites managers were conducted. In the case of Boston, Program Directors at the Boston Chinatown Neighborhood Center, the Center for Teen Empowerment, Discovering Justice, G-ROW, Generations, Inc., Junior Achievement, Steppingstone Foundation, and Roca, Inc. were interviewed. 

Web-Based Searches: Internet web-based searches of government and city and state level web sites were also conducted. In the case of Boston, important supplemental evaluation information was obtained from The After School Project: Boston’s After School for All Partnership, Mayor Menino's Boston 2:00-to-6:00 After-School Initiative, the Massachusetts Department of Education, the Boston Youth Zone and afterschool.gov, which provides access to government resources that support after school programs.

National Umbrella Organizations: To obtain additional evaluation information, a series of searches of national umbrella organizations were conducted. These Internet, web-based searches of national umbrella associations included Boys and Girls Clubs, Parents United for Child Care, Extended Service Schools Adaptation Initiative, Mass 2020, the National Institute on Out-of-School Time, Harvard Graduate School of Education’s Program in Afterschool Education and Research, Promising Practices in Afterschool, and Connect For Kids. These searches yielded important evaluation studies, often multi-site studies, which may or may not have had Boston as a site.

Evaluation Research Databases:  Evaluation information was also obtained from evaluation research databases. These research databases included but were not limited to the Harvard Research Family Project, SAMHSA, MDRC, Public/Private Ventures, Mathematica, Louis Harris and Associates, and the Promising Practices Network.  Evaluations completed by Policy Studies Associates yielded important information for Boston.

Peer-Reviewed Journals: We also used several databases containing evaluation research articles to complete an evaluation literature search. The data bases used include PsycINFO, Sociofile, EBSCO, and Web of Science. This literature search allowed us to identify several well-known journals that contained articles on after-school program evaluation. Some of the journals used in this search include Prevention Science and the Journal of Urban Health. The evaluation reports used in these articles were identified, sourced, and incorporated into the program evaluation summaries.