This page contains the meeting materials for the analyze stage of the AM-ME (adapted measure of math engagement) project. The essence of the analyze stage is to make meaning of data with your research partners. That is, incorporating partners’ perspectives and ideas as opposed to researchers doing all the interpretation. You will notice that we almost always host a drop-in meeting (1 hour, online, focused on presenting results/information) before convening our large group meeting (2 hours, in person, focused on discussions and interactive activities). This is because the analyze stage involves a lot of data/results/findings, and we found it helpful to let people sit with and take in the data before discussing or doing an activity about it. You will also notice that some of the information presented in these meetings refers to information from the plan and implement stages (e.g., who we collected data from, how the survey was administered); that is because these stages build directly on each other.
Data is central to the analyze stage; without data, there is nothing to analyze. Data, however, is not objective—how it is presented can greatly affect how it gets interpreted. This makes the behind-the-scenes activities (e.g., data preparation, data visualization) critical to how the analyze stage will go. You can find resources for those activities in analyze qualitative data, analyze quantitative data, and interpret findings.
Helpful Tips
- Interpreting results could be intimidating, especially for students who are often told that their answer is “wrong.” So, use open-ended questions. For example, “Why do you think we are seeing this result?” as opposed to “Do you think this is because of XYZ?”
- People process information at different rates. For meetings that are information dense, encourage partners to take notes. Consider providing prompt questions to guide their note taking (e.g., What results surprised you? What is unclear? What other information would be helpful?).
- Provide time and space outside meetings for people to process information. For example, at the end of an information dense meeting, consider asking “What other information do we need for our next week? What information would be helpful?”
- Consider using handouts to facilitate small group discussions. Handouts could feel more actionable than slides.
- Rather than jumping right into going over the results or analyzing the data, spend some time to orient people in terms of why we collected the data to start with as well as how the data speaks to other research activities in the project.
Year 1 qualitative and quantitative results preview drop-in meeting (July 2023)
This meeting focused on introducing the emerging themes from the first wave of interviews and focus groups, Student Focus Group Protocol - Spring 2023, and sharing descriptive results from the first survey that we administered, Math Engagement Survey Items.
Meeting length: 1 hour
- Facilitator Guide: Analyze - Year 1 qualitative and quantitative results preview drop-in meeting (July 2023)
- Slides: Analyze - Year 1 qualitative and quantitative results preview drop-in meeting (July 2023)
Year 1 qualitative results group meaning making meeting (July 2023)
This meeting focused on making meaning of results from the first wave of interviews and focus groups, Student Focus Group Protocol - Spring 2023. As a group, we moved themes into categories and began to define them.
Meeting length: 2 hours
- Facilitator Guide: Analyze - Year 1 qualitative results group meaning making meeting (July 2023)
- Slides: Analyze - Year 1 qualitative results group meaning making meeting (July 2023)
- Activity: Analyze - Math Engagement Qualitative Results Meaning-Making (themes to categories)
Year 1 quantitative results group meaning making meeting (July 2023)
This meeting focused on understanding the results from our first survey, Math Engagement Survey Items. First, in small groups, we gave feedback on each of the existing survey items. Then, we discussed how the survey items map onto (or do not map onto) the theme groupings that we identified so far.
Meeting length: 2 hours
- Facilitator Guide: Analyze - Year 1 quantitative results group meaning making meeting (July 2023)
- Slides: Analyze - Year 1 quantitative results group meaning making meeting (July 2023)
- Activity: Analyze - individual feedback to existing survey items As pre-work, we asked each research group member to reflect and note down their thoughts on each of the existing survey items. This helps them think about survey refinement.
- Activity: SAMPLE Copy of Survey Questions for What Exists, What is New, What Needs to Go Activity
- Activity: Individual feedback to the emergent theme groupings As post-work, we asked each research group member to reflect and provide their feedback on each of the theme groupings that we have identified so far.
- This meeting also covers the Plan stage of CEnR research because our group activity on refining the survey items directly contributes to future survey administrations.
Year 2 factor analyses results preview drop-in meeting (March 2024)
This meeting focused on going over the results from our initial adapted survey. We gave an overview of the exploratory factor analysis results, followed by Rasch analysis and Differential Item Functioning results. Half of the meeting was spent having a group discussion about one of the factors in more depth.
Meeting length: 1 hour
- Facilitator Guide: Analyze - Year 2 factor analyses results preview drop-in meeting (March 2024)
- Slides: Analyze - Year 2 factor analyses results preview drop-in meeting (March 2024)
- Activity: Analyze - EXAMPLE Copy of Factor Handout
Year 2 factor analyses results group discussion meeting (March 2024)
This meeting focused on going over the factor analysis and differential item functioning results from our initial adapted survey. Continuing with the activity that we started in Analyze - Year 2 factor analyses results preview drop-in meeting (March 2024), this meeting focused on discussing and refining each factor in more depth. We also brainstorm plans for the upcoming student focus groups, with the goal of gathering more information to refine the factors Student Focus Group Protocol - Spring 2024.
Meeting length: 2 hours
- Facilitator Guide: Analyze - Year 2 factor analyses results group discussion meeting (March 2024)
- Slides: Analyze - Year 2 factor analyses results group discussion meeting (March 2024)
- Activity: Analyze - EXAMPLE Copy of Factor Handout
- This meeting also covers the Plan stage of CEnR research because we discussed ways to refine the factors by gathering more information through student focus groups.
Year 2 qualitative and factor results group discussion meeting (May 2024)
This meeting focused on going over the results from our year 2 student focus groups, which focused on gathering more information to refine the factors Student Focus Group Protocol - Spring 2024. In doing so, we also included some factor analysis results (i.e., the item-person maps and differential item functioning results) since they also help inform factor refinement. Finally, we had small group discussions on the “lonely items” (i.e., items that did not load onto any factor), which went back to results from factor analysis of the initial adapted survey.
Meeting length: 2 hours
- Facilitator Guide: Analyze - Year 2 qualitative and factor results group discussion meeting (May 2024)
- Slides: Analyze - Year 2 qualitative and factor results group discussion meeting (May 2024)
- Activity: Analyze - EXAMPLE Copy of Refining Factor Handout
Thematic coding meeting (October 2024)
This meeting introduced the thematic coding process, and we practiced doing so using shortened transcripts from qualitative data we collected previously in this project. Using the themes that research group members identified, we also discussed what topics we’d like to know more about that could help us further refine our survey before its final administration in the spring.
Meeting length: 2 hours
- Facilitator Guide: Analyze - thematic coding meeting (October 2024)
- Slides: Analyze - thematic coding meeting (October 2024)
- Activity: to practice thematic coding, we used real transcripts from this project, which contained identifiable information. To meet the same purpose of practicing coding, you can consider using an AI-generated fake transcript, for example: Fake Transcript
Year 3 qualitative results and survey refinement group discussion meeting (Dec 2024)
This meeting focused on going over the results from our year 3 student focus groups, which focused on gathering more information for specific factors such as math identity and classroom climate Student Focus Group Protocol - Fall 2024. Based on the results, we brainstormed ways to refine the survey.
Meeting length: 2 hours
- Facilitator Guide: Analyze - Year 3 qualitative results and survey refinement group discussion meeting (Dec 2024)
- Slides: Analyze - Year 3 qualitative results and survey refinement group discussion meeting (Dec 2024)
- Activity: EXAMPLE - Classroom Behaviors Summaries (Research Group members were presented with summaries like this and responded to a series of questions about factor refinement)
The Adapted Measure of Math Engagement Research Group includes six students (Antonio Chavira, Brianna Espy, Ryan Ombongi, Serrah Ssemukutu, Salma Ahmed, and Diamond Tony-Uduhirinwa), five teachers (Nathan W. Earley, Karina Mazurek, Kathleen Morgan, Karla Rokke, and Ashly Tritch), and five researchers (Marisa Crowder, Samantha E. Holquist, Diane (Ta-Yang) Hsieh, Claire Kelley, and Mark Vincent B. Yu). Researchers Alyssa Scott, Olivia Reyes, and Avalloy McCarthy also extensively contributed to this work. Bloomington Public School District leaders Betsy Hawes, Marcie Coval, Julio Caesar, and Rik Lamm provided support to this work. Jennifer Widstrand served as the project manager.
If you have questions about the Adapted Measures of Math Engagement project, please contact Principal Investigator Samatha E. Holquist at sholquist@childtrends.org.
This project is funded by the National Science Foundation, grant #2200437. Any opinions, findings, and conclusions or recommendations expressed in these materials are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.