Vermont STARS Validation and Evaluation Study 2015-2017

Vermont’s STep Ahead Recognition and Improvement System (STARS) is the state’s Quality Rating and Improvement System (QRIS). A QRIS is a framework for recognizing, improving, and communicating the extent to which early care and education programs and providers demonstrate specified indicators of high quality practice. A QRIS also clearly articulates steps programs can take to progress from compliance with basic licensing regulations to attaining high standards of quality. STARS was developed in 2003 and provides a quality framework for licensed and registered child care centers and preschools, family child care homes, and school-age programs. Vermont STARS awards points recognizing program practices in five arenas of quality: Regulatory History, Staff  ualifications, Families and Community, Program Practices, and Administration.

As a recipient of the Race to the Top – Early Learning Challenge Grant (RTT-ELC), Vermont committed to evaluating STARS as part of an ongoing process of continuous quality improvement and to understand how STARS is working to achieve better outcomes for Vermont children. Child Trends worked with the Vermont STARS Evaluation Committee to conduct a validation and evaluation study of STARS. The design of this study was informed by national research on QRIS validation approaches and included the following research activities:

• Quality arena and indicator crosswalk: Compared the concepts of quality included in STARS to other QRIS to identify aspects of quality not included in STARS and the implications for STARS;
• Distribution of points awarded: Used administrative data to analyze the distribution of the STARS points (which were associated with a set of quality practices that programs demonstrate) and the capacity of the rating structure to produce accurate ratings;
• STARS verification process: Examined the reliability of the STARS verification and rating process to determine the degree to which the rating process is fair;
• Participant feedback: Gathered data from mentors and providers participating in STARS to understand their experiences and inform continuous quality improvement; and
• STARS rating structure: Examined program quality using the Environment Rating Scale (ERS) to determine the extent to which the STARS ratings are accurate and meaningful.