Fostering healthy data culture in education
Alexander Kurz is quick to describe himself as a data person. As the data and research manager for Thriving Together in Phoenix, Kurz is always on the search for quantitative insights into how more students can succeed in school en route to their careers.
But even when Kurz discovered data insights that could help boost student performance, there was no guarantee of action. When heâd share his findings, many of the school and district leaders were wary and defensive even though the data wasnât specifically critical of their individual work.
âI had to learn a lot around relationship-buildingââa skill he admits heâs had to hone, says Kurz. Over time, he refined his approach to starting conversations with teachers and administrators about student-performance data.
Kurz is not alone, according to the discussion at a recent Tableau Data Fellows training. Selected from StriveTogetherâs nationwide network of 65 Cradle-to-Career programs, the 15 Data Fellows, Kurz included, are each working alongside educators to build a continuum of services and keep at-risk kids on course. Yet all shared experiences involving educators reluctant to analyze student- and school-performance data.
But why? Wouldnât the insights help struggling or at-risk students succeed?
Not everyone shares the enthusiasm over data, says Candace Simon, senior data analyst at Bridging Richmond in Richmond, Virginia.
âSchools are not that receptive when we initially say that we are working on their data, or we're working to support their data plans. Data culture, especially in the education setting, is usually met with a lot of tension,â Simon said.
âA Finger-Pointing Mentalityâ
As for a source of this tension, several point to the 2002 implementation of the No Child Left Behind Act and the use of standardized testing to measure adequate yearly progress toward achievement targets. At the time, NCLB was touted as a cutting-edge use of data in the education system. The law would reshape the education culture by holding teachers accountable for student outcomes, advocates believed.
But 14 years later, there are indications that NCLB has actually weakened the data culture in many school districts.
NCLB did help establish a more comprehensive and inclusive data infrastructure for districts, but few educators and administrators were educated on the use of data in their individual work. Since NCLB passed, the primary focus has been the end-of-year assessment of student performance.
That type of testing does little to inform ongoing instruction; it primarily affects teacher evaluation. And in the absence of data to monitor student progress throughout the year, post-school year evaluation was akin to turning the crank on a Jack-in-the-Boxâa loud surprise with little forewarning.
Tensions began rising almost immediately, in part due to negative evaluations resulting in pay freezes, skipped promotions, or even dismissals.
Geoff Zimmerman was there in the early days, working with educators and administrators in the greater Cincinnati area.
Now the interim executive director of StrivePartnership (the local StriveTogether member organization), he is supporting the StriveTogether team running the Data Fellows program with a sense for what went wrongâand what might still be possible.
âWhen I started this work 10 years ago, one of the first project was a data report card on outcomes across Cincinnati and Northern Kentucky,â said Zimmerman. â[A]t the time, there was more of a finger-pointing mentality and not a lot of talk at all about data being used to improve.â
While the original intent was accountability, heâs quick to point out that the data from this period went a long way in scoping the problem.
For the first time researchers could disaggregate the data by different subgroupsârace, gender, poverty levelâbut âbeyond admiring the problem, we didnât do much at the time to build understanding of how using data can improve decision-making,â Zimmerman said.
By most estimates, this accountability-only approach did not provide the intended boost to student performance, but it did show that no single role owned the task of improving student performance.
âWhat weâve since tried to do with the StrivePartnership is create this idea of shared accountability and differentiated responsibility,â Zimmerman said.
Weâre trying to build this mentality that we are all in this together, that these are our shared outcomes, and asking what we can collectively do to improve these outcomes.
Data as a Flashlight, Not a Hammer
Still, the ripples of the imbalanced rollout are still palpable to the Data Fellows.
âIn many, many cases, educators have been called out by virtue of data,â said Kurz. âSo, usually, the association with data is negative, and over time, this has built an aversion to data.â
Sarah Weppner of the Treasure Valley Education Partnership echoed her colleagues. Data has been used âmore for accountability than improvement,â said Weppner, director of continuous improvement at the Boise, Idaho organization. But she was also quick to point out the value in rebuilding trust by showing data âas a flashlight rather than a hammer.â
The flashlight-hammer analogy comes up often when you talk to the Data Fellows. Theyâve developed a support network of sorts for dealing with people who distrust data. And their efforts to rebuild a culture of data deliberately starts at the top.
Joe Adams, research coordinator with the Public Affairs Research Council of Alabama, says workshops helped build confidence and shared understanding with local superintendents.
Building a culture isnât something that happens overnight or can be adopted out of the box. It requires continued focus and principled decision-making to overcome varying degrees of defensiveness at every level.
For many, earning buy-in from senior leadership is a criticalâand often challengingâfirst step.
A recent McKinsey Global Survey found that organizations with high-performing analytics programs are nearly three times more likely than their low-performing peers to have executive sponsorship for their analytics program.
Comparing notes across the cohort, the fellows say the same is true in their school districts.
In his workshops, Adams incorporates data visualizations into an environment that gives the superintendents a chance to explore the information just as he would, drilling down into certain issues without judgment.
âWe let the principals go through their day, function by function, just to see where they stand,â said Adams. âA lot of them have been through the reports that the state provides, but this kind of gives a clearer picture of where they are and how much they've gained from the previous year.â
The process is not designed to lead to specific conclusions, but rather to unearth other questions. The data can show what happened, but it is still up to the people in the room to ask why.
âThe data gives us a chance to have powerful conversations about groups of children who might not be doing as well as they should,â said Adams.
Other Data Fellows described variations on the workshop concept. Many build âdata walksâ to allow educators to be equal participants in the analysis process rather than just the object of the findings.
While the style of a data walk can vary, the core concept is to get people up, walking, and engaged around a set of questions. Facilitators are not telling participants what to think, but taking them through the process and helping them come to conclusions together.
âWhen we were able to present the data first and show them some of the bright spots, we get more buy-in,â Simon said. âPeople can really see how they can use data to help their work.â
Allowing Data to Drive Conversations
Collaboration is a crucial piece of this culture-building, allowing people to quickly and easily share and build on each otherâs findings. The Data Fellows say it is important is to create a safe space where the educators can see themselves in the data and talk about their experiences. Theyâre more open about changes and where those changes could be seen in the metrics.
In doing so, the review and analysis of student-performance data becomes something that happens through teachers, not just to them during evaluations. Treasure Valleyâs Weppner has even seen a physical change in people as they go through the process.
âWe will have meetings where people are quite reserved. But then they get up for the data walks, and that's when the conversation really starts to generate,â she said.
By exploring data alongside educators in a trusted and secure environment, administrators can empower teachers to do what is necessary to help their students be successful.
âUltimately, I believe these conversations help to set up a strong factor analysis,â said Zimmerman. âOnce everyone has a shared idea of what we know, we can start looking into the strategies that are helping and that are hindering progress.â
In return, teachers have confidence that the district understands their reality as they make difficult decisions or advocate for increased resources.
These types of feedback loops are critical to rebuilding data culture and the trust needed for schools to not just survive but to thrive.
And with the Every Student Succeeds Act (ESSA) era dawning, educators at all levels have an opportunity to learn from the lessons of the NCLB experience.
Kurz is optimistic that change is possible. While working alongside teachers and administrators to dive deeper into the data, heâs seen attitudes quickly change.
âOnce we start digging, they realize they have a lot of data. Initially, we help them with that, but in the long term, we want to build capacity in the local sites so they can do it themselves. My hope is that I'm working myself out of a job,â he said.