Transforming School Systems Through Evaluation

Introduction

Evaluation is a term often talked about in education spaces in the context of school improvement, but what is an evaluation? Evaluations—used to understand and examine a program or set of practices used within school spaces—collect and analyze information about the activities, characteristics, and outcomes of that program and/or set of practices. Evaluations may also be defined as the systemic application of research methods to address questions about program operations and results. Therefore, the main goal of an evaluation is to learn how a school’s program(s) or practice(s) can be better implemented or improved.

Figure 1. Stages of an Evaluation

The figure’s circular structure shows that evaluations can be cyclical and iterative.

Considerations for Conducting Program Evaluations

School systems can use evaluations to learn about various programs and practices that school leaders, educators, and students use day-to-day. Therefore, evaluations can help you and your team answer questions like the following:

• Does a program or practice meet its intended goals and/or outcomes?
• Are the program or practice goals appropriate and useful?
• How can a program or practice be improved?
• Are there unintended consequences to a program or practice?

In addition to these questions, you should consider several key factors before diving into the evaluation process. These factors include:

• The “why” for conducting an evaluation
• The intended use of the evaluation
• The type of evaluation you will use (these types will be explained in more detail below)
• The time frame in which you must conduct your evaluation (e.g., short-term versus long-term timeline)
• Data sources for the evaluation (who you will collect data and information from and who you already have this information from)
• The evaluation’s audience (who will learn from or be engaged as part of the evaluation process)
• Who will lead your evaluation efforts and ensure that the process is conducted according to the laid-out plan
• What key parts of your school system are needed to carry out your evaluation

Evaluation Steps

To help you and your team address these questions and ensure that you understand the role of the key factors in your evaluation, we walk you through four key evaluation steps.

• Plan the evaluation: The first step in the evaluation process is planning, which includes gathering all the information and establishing the structures you will need to successfully conduct your evaluation. The planning stage should include time to determine the purpose of your evaluation (including your goals and audiences), develop a logic model, design the questions that will guide your evaluation, select your methods of collecting data and data sources, and establish the feasibility of the overall evaluation. At the end of the planning stage, you and your team should have a detailed plan and timeline. Child Trends has developed a data inventory toolkit that can help with the planning stage!
• Implement the evaluation: The next step is implementation, which includes conducting the evaluation plan laid out in step one. This entails collecting all the necessary data to achieve your school transformation goals and completing your evaluation within the planned timeline. In this step, it is important to refer to your timeline and logic model, which can be great accountability checkers that ensure the evaluation is unfolding as expected.
• Analyze the data collected: The third step entails analyzing the data collected and compiling all your learnings to prepare you for the final step. This step will require some kind of analysis, which can be qualitative or quantitative based on the types of data you collected in step two. Step three can also involve sitting with your team to discuss the data analyzed to make sure you are interpreting the data objectively. Check out our qualitative data toolkit to help you and your team with this step.
• Report and address next steps: The final step is reporting and next steps, which involve reflecting on your collected data and information, assembling your learnings, and sharing learnings with the community, participants, and any relevant stakeholders who can help with program and practice improvement. These stakeholders may include district/school leadership, as well as any team members interested in what was learned from your evaluation. This step also lays out next steps, which can involve repeating your evaluation or coming up with a plan to address your learnings.

The Continua of School System Evaluations

Now that we have discussed the general steps within evaluations, let’s touch on how evaluations can happen within a continuum. Even though there are specific types of evaluations and essential steps, you and your team can adapt an evaluation to your needs to ensure you gain the necessary knowledge and learnings.

The figure below demonstrates the continua on which your evaluation can exist to best meet your school system transformation goals. We will also walk you through each of the terms popularized by Newcomer, Hatry, and Wholey (2015), who published a Handbook of Practical Program Evaluation, which provides tools for evaluators to conduct an evaluation. The terms are demonstrated below to help you understand these additional considerations for ensuring your evaluation is useful and successful.

• Formative vs. summative evaluation. Formative evaluations are meant to be ongoing and evolve with the programs and practices of interest. Summative evaluations, on the other hand, typically happen at the end of a program or end of the school year to evaluate a practice, and typically take place at one time. Your specific evaluation can be either formative or summative, or some combination depending on what you hope to accomplish and learn from your evaluation.
• Participatory vs. objective observers. In your evaluation, your participants may be very engaged, as they would be in a participatory design; or, by contrast, they could be objective. As objective observers, your participants simply engage when called upon and do not engage in the design or interpretation of the learning, as would occur in a participatory design.

Figure 2. Continua of School System Evaluations

Source: Newcomer, K. E., Hatry, H. P., & Wholey, J. S. (Eds.). (2015). Handbook of practical program evaluation (Vol. 864). Hoboken, NJ: John Wiley & Sons.

• Qualitative vs. quantitative. One of the key parts of evaluations is collecting data and information. The type you collect is critical for what you hope to learn from participants. Qualitative based evaluations lean more towards narrative and stories from participants about their experiences. While quantitative evaluations, focus on surveys and collecting information in ways through which is can be quantified by numbers. Many evaluations use a combination of qualitative and quantitative methods to collect their data and information. The approach you select can depend on the time available you have, how the information collected may be better presented to your audience, and the overall purpose of your evaluation. A lot of value exists in both qualitative and quantitative approaches.
• Problem-oriented vs. non-problem-oriented. Evaluations can be used to learn a lot about your school system, but why you decide to conduct any evaluation may depend on whether you simply want to learn more or have a specific challenge you’d like to address. When designing your evaluation plan and determining its purpose, you will identify whether your evaluation is problem-orientated or not. Evaluations can have more than one goal, such as solving a problem and learning more about your school system. The goal(s) of your evaluation will ultimately shape how you design and conduct your evaluation.

In addition to these continua, remember that your evaluation can aim to be both a learning and accountability strategy. The learning piece can tell you how the program or practice is inspiring changes or challenges in your school system. The accountability piece can include updating funding partners on progress since implementing a program or practice(s) and on how a program or practice(s) is helping your school system abide by federal, state, and/or district policies.

Evaluation Types

There are six main types of evaluations used within education spaces: 1) developmental evaluations, 2) implementation evaluations, 3) outcome evaluations, 4) participatory evaluations, 5) formative evaluations, and 6) summative evaluations. You can use more than one evaluation type or adapt/combine evaluations to achieve desired learnings about your school system or classroom. The table below outlines each type of evaluation and when you might use it in your education work.

Table 1. Types of Evaluations for School Transformation

See more about evaluations here.

Logic Models and Evaluations

Depending on the type of evaluation(s) you and your team decide to conduct, it may be useful to develop a logic model to anchor how your evaluation(s) fit into your school system’s overall mission or goals. A logic model is a graphic display of program inputs, outputs, and outcomes that also identify external factors that may impact a program’s ability to achieve its desired outcomes and guide the development of evaluation activities. A logic model is a very useful resource for facilitating communication and all the moving parts of program, practice, and evaluation. This linked resource can help you and your team develop a logic model and a toolkit. We also walk you through the components of a logic model so you can create your own.

• Inputs: When creating a logic model, always begin with your inputs. Inputs consist of anything that is needed to engage in the program or practice you are aiming to evaluate, including 1) personnel resources, which can consist of program managers and participants; 2) financial resources, which can include sources of program funding; 3) technology resources like computers or tablets; 4) infrastructure resources such as physical spaces; and, finally, 5) program resources such as curricula or materials associated with a program or practice.
• Activities: Program or practice activities consist of anything that must happen or be done to ensure implementation of the program or practice being evaluated. Examples of activities can include meetings with group leaders, group meetings with the program team, trainings for students and educators, and program sessions.
• Outputs: Program or practice outputs consist of who is impacted by the activities listed in the previous bullet. Outputs might include students’ feeling of empowerment or educators having tools that help them in their classrooms. This link has examples that can help you determine your outputs and how they may compare to outcomes.
• Short-, medium-, and long-term outcomes: Outcomes focus on specific results that participants gain by being a part of a program or practice. Outcomes in a logic model are usually categorized by short-, medium-, and long-term outcomes. Short-term outcomes are those that occur shortly after participation (e.g., immediately) and often include changes in knowledge, skills, attitudes, or awareness among participants. Medium-term outcomes are those that occur because of short-term outcomes (e.g., after a few months or within the first year of participation). An example of a medium-term outcome would be knowledge gained from participating in a program leading to improved practices in the classroom. Finally, long-term outcomes are the broader impacts of participation in the program or practice. Long-term outcomes are usually a cumulative effect of short-term and medium-term outcomes—for example, a program or practice might reduce chronic absenteeism in a school over a few years.

Figure 3. Example Logic Model Template

Source: Newcomer, K. E., Hatry, H. P., & Wholey, J. S. (Eds.). (2015). Handbook of practical program evaluation (Vol. 864). Hoboken, NJ: John Wiley & Sons.

Conclusion

This practice guide is intended to introduce school improvement evaluations and provide basic guidance to help you and your team conduct the right type of evaluation. To improve your school spaces, you must create opportunities to learn—exactly what evaluations offer. Take the opportunity to use evaluation processes that can help you improve programs and practices in your schools to better serve teachers, students, families, and communities.

Resources

• Comprehensive Resource Library for Leading PreK-12 Education Systems Change – Development (Child Trends, 2025)
• Steps to Guide Sustainable Change in School Systems (Samantha Holquist, Lorena Aceves, Mara Schanfield, and Olivia Reyes, Child Trends, 2025)
• Program Evaluation Toolkit (Regional Education Laboratory Central, Institute of Education Sciences, 2021)