I’m not very far into my journey towards a master’s in educational leadership, but I’ve already heard about “data-based decision making” enough to know that it’s a big deal. One of the three courses I’m taking this summer is about research methods specifically, and the course I’ll be taking in the fall is about “leading with data”.
I don’t have to be convinced that using data to make decisions is important. I teach math and science – I love data. While I teach my students the ins-and-outs of designing properly controlled experiments, I also know that my own “experiments” – that is, the assessments I use to test my hypothesis about what I think they have learned – are much harder to control. So, I understand that when analysing the data from assessments, I need to keep in mind that there are always potential sources of error.
We dealt with this in great detail in my research methods class. We looked at a range of research methods, like surveys, focus groups, archival data, case-studies and achievement data, and discussed the benefits and limitations of each. We talked in great detail about how to collect data and then how to interpret it while bearing in mind the inherent limitations and potential biases of the research method(s) employed. This is all really important stuff.
The thing is, teachers collect data all the time. Rather than going out to find data, we are constantly bombarded with it. We ask questions, make observations, collect anecdotal data and student achievement data every hour of every day. Not only are we always collecting data, we are always making decisions. Where should I stand? Which student should I call on? Is it okay for two friends to be working together? Can they have an extension on the homework?
I know that these day-to-day decisions are not the ones people talk about when the term “data-based decision making” is tossed around. And I know that with the amount of data teachers receive any given day, and with the number of decisions that they have to make on-the-spot, a thorough analysis of the biases and limitations in our data set is impossible; however, I do wonder, with the rapid pace at which we operate, how much of our data collection is incidental, and how much is intentional?
- When we decide to recap the lesson on factoring for the third time, it is because most kids need a review, or because the most vocal kids need a review?
- When we interview kids to find out about an incident on the playground, does most of our information come from extroverts?
Do you have a strategy for managing the bias in your daily data set and daily decisions? If so, please comment.