Being Data-Informed vs Data-Driven
Within the past week or so, I’ve encountered two people who have purposefully distinguished between being “data driven” and “data informed.” First, Rick Saporta differentiates these two approaches in his recent presentation on business data strategy. And second, Tiankai Feng situates both “data informed” and “data driven” decision making on a continuum in his book Humanizing Data Strategy (p. 16-17).
Although they make slightly different points, the heart of both Saporta and Feng’s positions is that “data-informed” decision making entails explicitly combining personal expertise with insights gained from data, whereas “data driven” decision making entails making a decision entirely based on data. In Feng’s argument, these decisions are entirely data-driven because they are made autonomously by some model or algorithm. In Saporta’s, these “data-driven” decisions are still made by people, but they’re made by people who either fail to appropriately weigh their expertise alongside the available data or who don’t have sufficient expertise to factor into the decision at hand.
This is an interesting distinction to me, both because everything in contemporary education anymore must be “data driven,” and because I’ve written before in this blog about making “data driven decisions.”
To some extent, the distinction is arbitrary. Or, maybe not the distinction itself, but the labels at least. In my previous post about my team’s data-driven decision framework, I’m actually advocating for what Saporta and Feng would call “data-informed decision making.” Am I going to go back and edit that post and all of my team’s document templates now to reference “data-informed decision making” rather than “data-driven decision making?” Zero chance. If, at some point in the future, the “data driven” and “data informed” labels end up being proper nouns that everyone agrees on and are codified in the Oxford English Dictionary and the distinctions between them are ontologically critical, then sure, I’ll update my usage. For now though, I think I’ve happened to stumble on two people who kinda happen to use a common vocabulary to make similar distinctions.
That said, although I don’t think the labels themselves matter all that much, the approaches they represent matter quite a bit. In education, the vast majority of our decisions should be what Feng and Saporta refer to as “data-informed” – shaped by both personal expertise and relevant data. There are some cases where “data-driven” decisions make sense – mostly low-stakes micro-decisions where the opportunity cost of having to bring in an expert to help make the decision isn’t worth the potential benefit that expertise adds. Consider a math app that automatically generates problems for students based on their prior performance. Could a teacher with deep knowledge of math and of her students come up with better questions? Probably! But is continuously generating the best-possible practice problems the best use of her time? Probably not!
But for medium- or high-stakes decisions about students, teachers, or schools, I think we absolutely need to take the “data-informed” approach. Partially because no database or student information system is going to store quantitative data representing all of the relevant contextual factors we ought to consider when making these decisions – even state-of-the-art AI models can’t fully understand your unique context. And partially because, on principle, it’s Orwellianly dystopian to outsource that shit to an algorithm.
Regardless, even if I’m not going to rebrand all of my own stuff from “data driven” to “data informed,” I do think the distinction made by Saporta and Feng is worth using as a lens to inform how we think about using data in our work.
If you’re enjoying reading these posts, please consider subscribing to the newsletter by entering your email in the box below. It’s free, and you’ll get new posts to your email whenever they're published.