Program

Nov 05, 2010

OVERVIEW

Quantum
Opportunities Program (QOP) is an after-school program providing intensive case
management, mentoring, supplemental education, developmental activities,
community service activities, support services, and financial incentives for
at-risk youth entering high school. Participation in QOP was primarily aimed at
increasing high school graduation rates and postsecondary education and training
enrollment. Secondary aims of the program included improving high school
academic grades and test scores and reducing engagement with risky behaviors
such as substance abuse and teen parenting. The 5-city pilot study revealed
participation in QOP produced significant impacts on increasing academic skills,
receipt of honors and awards, high school graduation rates, college attendance,
and community service; additionally, participation in the QOP program decreased
rates of high school dropouts and teen parenting. Findings from the 7-city final
impact study of the demonstration program revealed several short-term impacts,
but no long-term impacts on increasing the likelihood
of graduating high school with a diploma or earning a GED. There were no impacts
on improvements in test scores, academic grades, or credits earned. There were
also no impacts on either enrolling or completing postsecondary education or
training. In addition, there were no impacts on employment-related outcomes, and
there were no impacts on risky behaviors, physical or mental well-being, or
family life. Some positive impacts were found within specific sites, however.

DESCRIPTION OF
PROGRAM

Target
population:
 At-risk students entering 9th grade.

Quantum Opportunities Program (QOP) was designed as an after-school program to
provide an intensive package of “quantum opportunities” and services to
participants throughout their four years in high school. Participation in QOP
was aimed, primarily, at increasing high school graduation rates and
postsecondary education and training enrollment. Secondary aims of the program
included improving high school academic grades and test scores and reducing
engagement with risky behaviors such as substance abuse and teen parenting.

The program was originally piloted in 1989 at 5 locations, including:
Philadelphia, PA; San Antonio, TX, Saginaw, MI; Oklahoma City, OK; and
Milwaukee, WI. Students were eligible for participation if they were graduating
8th grade and were from families receiving public assistance. The
program lasted 4 years.

In 1995, the US
Department of Labor and The Ford Foundation funded a QOP demonstration program
that included 7 locations: Cleveland, OH; Fort Worth, TX; Houston, TX: Memphis,
TN; Washington, DC; Philadelphia, PA; and Yakima, WA. The Ford Foundation funded
the Philadelphia and Yakima sites, while the US Department of Labor funded the
remaining 5 sites. Students were eligible for participation if they were
entering high school with a GPA below the 67% percentile and the high school had
at least a 40% dropout rate. The QOP demonstration
program served youth at the beginning of 9th grade in 1995 and
followed them through 2000, for a total of 5 years.

Community-based
organizations were responsible for operating the QOP demonstration program in
each location. The QOP model comprised 4 components: case management and
mentoring, educational activities, developmental activities, and community
service. Other components included financial incentives, including stipends, and
supportive services such as snacks and transportation. More specifically,
educational activities comprised one-on-one tutoring and computer-assisted
instruction catering to meet specific needs as well as general assistance in
reading and mathematics; field trips to local colleges were also included to
raise awareness of postsecondary opportunities. Developmental activities
addressed risky behaviors and promoted cultural awareness and provided
recreational activities. Community service activities involved visiting local
nursing home residents or volunteering at local food banks and were intended to
help youth develop feelings of responsibility. Additionally, with the case
management component built into the QOP model, referrals were also provided such
as mental health services and summer jobs programming.

Students were
allocated to receive up to 750 hours of services per year, equally divided among
educational services, developmental activities, and community service. Among
program administrators, students were considered to be enrolled in the program
even if they transferred to another school, dropped out of school, were
incarcerated, or were inactive in QOP-related activities. All services continued
throughout the school year, including summer months.

Total QOP
expenditures per enrollee averaged $25,000 over the 5-year demonstration
program; however, expenditures ranged from $18,000 to $22,000 among the US
Department of Labor sites, while expenditures were higher among the two Ford
Foundation sites, Yakima ($23,000) and Philadelphia ($49,000). These figures do
not include technical assistance services, which were provided through
additional funding from the Ford Foundation.

EVALUATION(S)
OF PROGRAM

Study 1: Hahn A,
Leavitt T, Aaron, P. (1994). Evaluation of the Quantum Opportunities Program:
Did the program work?
 Waltham, MA: Brandeis University, Heller Graduate
School.

Evaluated population: 50 students from 5 sites: Philadelphia, PA; San
Antonio, TX; Saginaw, MI; Oklahoma City, OK; and Milwaukee, WI. Students were
eligible if they were leaving 8th grade and their families were
receiving public assistance.

Approach: In the summer of 1989, a recruitment of students about to enter
ninth grade was undertaken. In each of the 5 sites, 50 students were randomly
selected from lists of eighth-grade students whose families received public
assistance. Each student was then assigned to either the QOP group (n=25) or
control group (n=25). This study compared the baseline (9th grade) to
post high school, with approximately 15% attrition.

Throughout high school, both groups completed questionnaires about demographic
characteristics, work and school experiences, health knowledge, and personal
attitudes and opinions. They also completed the Test of Adult Basic Education to
assess academic skill levels and the APL 40-item Version Survey-CCP Tier Mastery
Test to assess functional skill levels. Post-high school follow-ups were
conducted by telephone and mail surveys. Over the course of the program,
participants spent an average of 1300 hours participating in the program.

Results: Two years elapsed before impacts of QOP began to show. After two
years, the experimental group averages for all 11 academic and functional skills
were higher than the control group, with 5 of the scores statistically
significant. By the time the sample was ready to finish high school, the
experimental group was significantly higher on all 11 skill scores. Also, after
two years, the experimental group’s educational expectations were significantly
higher than the control group.

In the post-high school period, the experimental group was significantly more likely to have graduated from high school and to be in a post-secondary environment, thus, much less likely to have dropped out than students in the control group. The experimental group was three times as likely to attend a four-year college and twice as likely to attend a two-year institution as the control group. The experimental group was also less likely to have children, more likely to have received honors and awards, and to have performed community service. Program impacts were much stronger for the Philadelphia site than the other sites. In all reports, the Philadelphia site tended to be more successful, especially with the high school graduation and post-secondary attendance percentage. In some instances, Philadelphia was the only site producing statistically significant results. It was speculated these results may have been due to the fact that other sites had a smaller number of children and/or the other sites were unable to implement and perform the program similar to the Philadelphia site.

Study 2: Maxfield M, Schirm A, Rodriguez-Planas N. (2003). The Quantum
Opportunity Program Demonstration: Implementation and Short-Term Impacts.
Washington, DC: Mathematica Policy Research, Inc.

Evaluated population: 9th grade students with low academic grades attending high schools
with high dropout rates in the following 7 US cities: Cleveland OH (n=100); Fort
Worth, TX (n=100); Houston, TX (n=100); Memphis, TN (n=100); Washington, DC
(n=80); Philadelphia, PA (n=50); and Yakima, WA (n=50). Except for the Yakima
site, the QOP demonstration program schools primarily served black or Latino
students in urban neighborhoods. Poverty status, particularly free lunch
eligibility, varied among the demonstration program school sites.

Approach: All
eligible youth were randomly assigned to the QOP intervention group (n=580) or
the control group (n=489). This study measured programmatic implementation and
short-term outcomes–i.e., at the fourth and fifth years of the program and prior
to program completion. A first round of in-person interviews was conducted
during the fourth academic year, and asked questions around topic areas such as
risky behaviors (substance use, gang activity, arrests, and sexual behavior) and
resiliency (presence of influential adult, optimistic outlook on future, and
belief risky behaviors are wrong); students were also administered reading and
mathematic achievement tests. Approximately 8 months later, a telephone survey
was conducted asking participants about high school graduation, postsecondary
activities, risky behaviors, and, additionally for the QOP-enrolled group,
attitudes towards QOP. Academic transcripts were also requested.

Results: QOP-enrolled students, on average, spent 174 hours in QOP-related
activities–falling short of the anticipated goal of 750 hours per student. There
was also variation in program implementation among each site, as community-based
organizations found implementation difficult due to its intensive focus, and
both funders allowed flexibility when adopting and implementing the QOP model.

Among QOP-enrolled students, there was a significant
impact on the likelihood of graduating high school compared with the control
group. However, there were mixed impacts on postsecondary education or training
outcomes. For example, while there were significant impacts on postsecondary
training or having a job with health insurance, there were marginal impacts on
attending a 2- or 4-year college, and no impacts on attending a four-year
college. There were also mixed impacts on resiliency outcomes. For example,
while there were significant impacts on QOP students’ perceptions of
participating in a program that helps them make good decisions in life, there
were no impacts on QOP students’ perceptions of having an influential adult in
their life compared with the control group. There were no impacts on
improvements in test scores, academic grades, or credits earned. Additionally,
there were no impacts on reducing risky behaviors, and in fact, increased
behaviors in a handful of areas, including substance use and binge drinking. The
authors stress these findings represent short-term impacts, as these data were
collected during the fourth and fifth year of the demonstration program when
many students were still enrolled in high school or recently graduated.

Study 3: Schirm
A, Rodriguez-Planas N. (2004). The Quantum Opportunity Program Demonstration:
Initial Post-Intervention Impacts. Washington, DC: Mathematica Policy Research,
Inc.

Evaluated population:Please see Study 2. At the time of this post-intervention follow-up study, study
participants were, approximately, 21 or 22 years of age.

Approach: This
study measured post-intervention impacts, approximately 2 years after the
completion of the demonstration program. A brief telephone survey collected
information on 4 domains: high school completion, postsecondary attainment,
postsecondary activity, and risky behaviors and family life. High school
completion included high school graduation or GED receipt. Postsecondary
attainment included a cumulative attainment of college, vocational training,
apprenticeships, or military. Postsecondary activity included engagement in
postsecondary education and training or employment. Risky behaviors and family
life included substance use, recent criminal activity or involvement with
criminal justice system, teen and/or single parenting, and welfare receipt.

Results: There
were no significant impacts increasing the likelihood of graduating high school
with a diploma or earning a GED. There were mixed impacts on postsecondary
attainment. For example, QOP-enrolled participants were significantly more
likely than the control group to have attended college or vocational or
technical school, enrolled in an apprenticeship, or enlisted in the military;
however, there was only marginal significance among QOP-enrolled participants,
compared with the control group, of ever attending a 2- or 4-year college; and
there was no impact of having completed any 2- or 4-year college. QOP-enrolled
participants were significantly less likely to be employed full-time with health
insurance coverage. There were no impacts improving high school grades or
achievement test scores. Additionally, there weremixed impacts on risky
behaviors and family life. For example, QOP-enrolled participants were
significantly less likely than the control group to use illegal drugs in the
past month; however, there were no impacts on decreasing the likelihood of teen
parenting, binge drinking, committing a crime, or being arrested or charged with
a crime.

Subgroup analyses
by gender did not reveal consistent beneficial impacts for one gender over the
other. However, the QOP program appeared to produce beneficial impacts among
younger students (i.e., those entering the program at 14 years of age or
younger) compared with older students (i.e., entering the program at 15 years of
age or older), particularly with postsecondary school outcomes. There was little
evidence that the program consistently impacted students by grade distribution.
Varying impacts by QOP site were also observed.

Study 4: Schirm
A, Stuart E, McKie A. (2006). The Quantum Opportunity Program Demonstration:
Final Impacts. Washington, DC: Mathematica Policy Research, Inc.

Evaluated population: Please see Study 2. At the time of the final impact study, participants were,
approximately, 23 to 25 years of age.

Approach: This
study measured final impacts, approximately, 4 years after the completion of the
demonstration program. Again, participants were administered a telephone survey
collecting information, including high school completion, postsecondary
attainment, postsecondary activity, risky behaviors and family life, as well as
detailed information about employment.

Results: The
implementation evaluation, as noted, found that the QOP model was not well or
fully implemented, and participation fell below intended levels.

No impacts were found on the likelihood of graduating
high school with a diploma or earning a GED. There were also no impacts of
either enrolling or completing postsecondary education or training, including
college, vocational or technical school, apprenticeship, or the military. In
addition, there were no impacts on employment-related outcomes, as QOP-enrolled
participants were no more likely to be employed, unemployed, or have a job with
benefits such as health insurance or a pension. Furthermore, there were no
overall impacts among risky behaviors, physical or mental wellbeing, or family
life. There was one adverse impact, where QOP-enrolled participants were
significantly more likely than the control group to have been arrested or
charged in the past 2 years; however, there were no significant differences in
binge drinking, using an illegal drug, self-reported health status,
childbearing, or receiving welfare benefits.

Subgroup analyses were also performed by gender, age,
student grade distribution/GPA, and site location. Participating in the QOP
program revealed no consistently beneficial impacts favoring one gender over
another. However, while the QOP program appeared to positively impact younger
students (i.e., those entering the program at 14 years or younger) compared with
older students (i.e., those entering the program at 15 years of age or older).
There was also little evidence that the program consistently impacted students
who varied in terms of grade distribution/GPA. Occasional negative impacts
were also found. Varying impacts by QOP site were also observed, with the
Cleveland, Washington, D.C., and Philadelphia sites each having several positive
impacts. Most site impacts became non-significant when adjustments were made
for the number of analyses.

Costs ranged across sites from $18,000 to $49,000 per
enrollee across the five years of the program.

SOURCES FOR MORE INFORMATION

References:

Study 1: Hahn A, Leavitt T, Aaron P. (1994). Evaluation of the Quantum
Opportunities Program: Did the program work?
 Waltham, MA: Brandeis
University, Heller Graduate School.

Hahn A. (1994). Extending the time of learning. In
D.J. Besharov (Ed.), America’s Disconnected Youth: Toward a Preventative
Strategy
, pp. 233-266. Washington, DC: CWLA Press and American Enterprise
Institute for Public Policy Research.

Study 2: Maxfield M, Schirm A, Rodriguez-Planas N. (2003). The Quantum
Opportunity Program Demonstration: Implementation and Short-Term Impacts.
Washington, DC: Mathematica Policy Research, Inc.

Study 3: Schirm A, Rodriguez-Planas N. (2004). The Quantum Opportunity Program
Demonstration: Initial Post-Intervention Impacts. Washington, DC: Mathematica
Policy Research, Inc.

Study 4:
Schirm A, Stuart E, McKie A. (2006). The Quantum Opportunity Program
Demonstration: Final Impacts. Washington, DC: Mathematica Policy Research, Inc.
Program also discussed in the following Child Trends publication(s):

Manlove, J., Terry-Humen, E., Romano Papillo, A.,
Franzetta, K., Williams, S., & Ryan, S. (2002). Preventing teenage pregnancy,
childbearing, and sexually transmitted diseases: What the research shows

(Research brief). Washington, DC: Child Trends.
Manlove, J., Terry-Humen, E., Romano Papillo, A.,
Franzetta, K., Williams, S., & Ryan, S. (2001). Background for
community-level work on positive reproductive health in adolescence: Reviewing
the literature on contributing factors.
 Washington, DC: Child Trends.

Redd, Z., Brooks, J., & McGarvey, A. (2002). EducatingAmerica‘s youth: What makes a difference (Research brief). Washington , DC : Child
Trends.
Redd, Z., Brooks, J., & McGarvey, A. (2001). Background for community-level work on educational adjustment in adolescence:
Reviewing the literature on contributing factors
. Washington, DC: Child
Trends.
Zaff, J. F. & Michelsen, E. (2002). Encouraging
civic engagement: How teens are (or are not) becoming responsible citizens

(Research brief). Washington, DC: Child Trends.
Zaff, J. F. & Michelsen, E. (2001).Background for
community-level work on positive citizenship in adolescence: Reviewing the
literature on contributing factors.
 Washington, DC: Child Trends.

KEYWORDS:
Adolescents (12-17), Youth (16+), Young Adults (18-24), High School, College,
High-Risk, School-based, Community-based, Skills Training, Mentoring, Tutoring,
After School Program, Service Learning, Vocational Learning, Case Management,
Teen Pregnancy, Job Skills, Employment, Aggression/Violence/Bullying,
Delinquency, Alcohol, Marijuana, Illicit, Prescription Drugs, Life Skills,
Self-Efficacy, Community Service, Helping/Social Responsibility, Academic
Achievement, High School Completion, College Enrollment/Preparation

Program
information last updated 11/5/10