Imagine for a second you are leading an organization.
One morning, an email arrives in your inbox with a report attached.
The report includes a graph of personnel data showing one group is disproportionately underserved compared to the rest of the staff based on hair color.
The report shows individuals with red hair are systematically excluded and mistreated relative to everyone else in the organization.
Next, imagine you received this report every month for a year and the data remains unchanged each time you viewed it.
What would do?
How might you respond differently after reading these monthly reports?
What could prevent you from taking action?
How would you feel?
A few years ago, I was working on a study1 exploring this exact scenario (minus the red-haired part).
What are the effects of sharing equity-focused data reports with school leaders?
The study ended up highlighting a need for me to think differently about how to share and present data, especially when it sheds light on inequities (more on this in a minute).
What we did
Let’s go to Eugene, Oregon, and the University of Oregon where along with football, the Ducks are well known for producing a lot of educational research.
One day, my advisor proposed an idea that immediately caught my attention, one that was uncommon in educational research — instead of validating the merits of a school practice, he wanted to debunk one.
The basic idea:
To test whether sending school leaders monthly reports showing racial disparities in their school’s discipline data would trigger any measurable actions.
Note: If you are reading (or listening) to this story and don’t work in school, imagine reviewing the results of an audit or program evaluation showing a group you serve is routinely excluded from key meetings or decisions.
The set up
Our team recruited 35 school leaders from three school districts in the Pacific Northwest to participate in a double-blind, randomized control study (primer on this study method here).
To compare apples to apples, the administrators & schools all used the same data system (i.e., SWIS) to track student discipline referrals.2
Next, school leaders were randomized into two groups to rule out personal characteristics (e.g., age, race, gender, and years of experience). Next, schools were paired up based on disciplinary referral rates.
Control group: 17 leaders received a monthly Summary Report including figures and tables showing general patterns of discipline referrals.
Intervention group: 18 leaders received a monthly Equity Report including figures and tables showing Black v. White racial disparities.
Google images
What we measured
A year later, we looked a three measures that were accessible to us through each school’s data systems and district websites.
a. Report viewing (graphs and tables) we shared with administrators each month.
b. Discipline rate (Black-White rate difference). A rate difference of 0 indicated perfect equity. Anything above 0 indicated higher rates of discipline assigned to black students. Most schools disciplined Black students 2x as much as White students.
c. Annual school improvement plans describing behavior and academic goals compared before and after the intervention year.
One year later…
While we found out that school administrators in the intervention group ended up viewing MORE discipline reports than the control group, there was no significant change in disciplinary equity or goal setting.
The data suggested sharing monthly equity-focused data reports with school leaders to spur action was pretty ineffective.
Practice debunked.
So, what did this study reveal?
This was a small study but it did seem to call attention to a few things:
Sharing data (as a standalone strategy) such as data dashboards, slides, spreadsheets, or white papers can raise awareness, but is likely not enough to trigger action.
Organizational leaders reckon with data every day that may be uncomfortable, polarizing, or reveal truths that are hard to address or acknowledge.
Data can be viewed as punitive, shaming, or overwhelming to the people you most want to enlist in a change-oriented initiative within an organization.
What do you do if you receive a red-haired report?
The first red-haired report I remember sharing was something called the Washington Youth Survey.
Students in 6th-12th grade answered questions about mental health, substance use, safety and violence, and related risk and protective factors every two years in Washington State.
When I got the results back, I anxiously prepped and shared slides for a 15-minute portion of a school staff meeting. I expected the staff to be surprised or concerned to see an uptick in students reporting higher-risk behaviors from the previous year.
When the day of the faculty meeting arrived, the principal invited me to share a summary of the results. I talked through 5 graphs and tables for 10 minutes and then left the last 5 minutes for Q&A.
“Does anyone have any questions?” I recall asking.
Then…
Crickets and an awkward silence…Ugh.
Ah…this must be a red-haired report
Here’s what I’ve learned can help with sharing red-haired reports or any information that could be viewed by your audience as disruptive, discomforting, challenging, or even divisive.
Make the WHY explicit & transparent
It can help to cultivate a shared understanding of WHY people are looking at the data.
High-risk behaviors, race, patterns in school discipline, and staff performance are all highly subjective & PERSONAL topics.
So, before sharing data, try to prime your audience or team by asking them to reflect on their WHY individually and collectively for looking at a graph or trend line (e.g., I’ve seen groups highlight positive values such as human potential or interdependence).
Keeping the WHY front and center can help override deficit thinking, negative cultural narratives, or stereotypes that may be deeply embedded in how they read and interpret the data shared.
Try adopting or exploring a set of commitments BEFORE looking at a red-haired report:
Here’s one example (see more at the link below image)
It can help to remind yourself or others that the data did not create the disparities or trends you are seeing.
But, data can be used strategically to question the structures, norms, or policies that make race (or hair color) predictive of whether a student gets disciplined, a staff member gets promoted, or a colleague get invited to a cabinet meeting.
Framing (e.g., storytelling)
“Nice graphs. But, the numbers aren’t telling me a story that matters.”
This is likely the response I was getting when I shared the data slides in a middle school staff meeting. I’m sure many teachers were eager for me to wrap up my “talk” so they could go home.
Unless the numbers you share are embedded in a story you have framed, your audience will use the stories they already know to make sense of them.3
The Frameworks Insitute has been exploring conversations and strategies around data & social change for decades.
Here’s a useful question I found in their work that I’ve applied to my own:
What is the story that numbers could be used to tell that allows people to see solutions?
Here’s a link to a video on the Frameworks Institute’s website to give you a sense of their approach:
https://www.frameworksinstitute.org/publication/fast-frames-episode-2/
Pair data with support & accountability
Too often when I have shared data or had data shared with me, it can feel like someone is calling attention to a major personal failing or shortcoming and then expecting things to magically get better once I admire the problem.4
Ironically, exposure to racial disparities can cause people to become more, not less, supportive of the very policies that create those disparities.5
To avoid the crickets or resistance that red-haired reports may elicit, consider sharing promising practices or accountability measures that will help people see and believe efforts and steps are being taken to change current practices.
The good news for educators is that there are multiple lines of research showing approaches to help schools improve racial equity in school discipline.
No need to shy away from the red-haired report.
Here’s one example.
Red-haired reports in your context
I am most familiar with what red-haired reports can look like in schools.
I am curious about what these reports may look like in your setting and how you have shared them with others.
I hope this post resonated with you even if you have never attended the “natural wonder” that is a middle school faculty meeting.
A reason for sharing this post was not to call out school leaders, but instead to offer them some additional support and ideas related to data sharing.
I also have nothing against red-headed people.
Similiar, I hope if or when a red-haired report ends up in your inbox or desk you now have additional tools to try out.
Based on my experience, data (in its many forms) can be a very helpful tool to improve equity in organizations, but it’s important like any tool, to consider how, when, and why to use it.
And, I’ve learned it’s good to be aware of the potential pitfalls of sharing data without first considering the context, knowledge, worldviews, and needs of the people you are sharing it with.
As always, if you have a comment or question on this or other posts, please reach out to me:
Or, feel free to leave a comment here:
Until next time, thanks for reading or listening.
Be well & stay curious,
Eoin
References
McIntosh, K., Smolkowski, K., Gion, C. M., Witherspoon, L., Bastable, E., & Girvan, E. J. (2020). Awareness is not enough: A double-blind randomized controlled trial of the effects of providing discipline disproportionality data reports to school administrators. Educational Researcher, 49, 533-537.
For Catholic School Educators, I believe some schools still use JUGs (i.e., Justice Under God) which are the same as discipline referrals.
https://www.frameworksinstitute.org/article/the-storytelling-power-of-numbers/
Hetey, R. C., & Eberhardt, J. L. (2018). The numbers don’t speak for themselves: Racial disparities and the persistence of inequality in the criminal justice system. Current Directions in Psychological Science, 27(3), 183-187.
Hetey, R. C., & Eberhardt, J. L. (2014). Racial disparities in incarceration increase acceptance of punitive policies. Psychological Science, 25, 1949–1954.
35: The red-haired report