Kathryn Parker Boudett sees beauty in using data wisely

Transcript

ANGELA SCAFFIDI:

Welcome to the Bastow Podcast, conversations with big-thinkers about the big questions in education and leadership today. I’m Angela Scaffidi.

Kathryn Parker Boudett is the Faculty Director, Learning and Teaching, at the Harvard Graduate School of Education. She’s an author, academic, consultant and thought-leader with a particular focus on using data to improve learning and teaching. Kathy’s also the Faculty Chair of the Data Wise Leadership Institute at Harvard. Kathy was at Bastow recently to introduce the Institute to more than 100 education leaders from across Australia and overseas. This is the first time Harvard has chosen to facilitate Data Wise outside of the United States. Keep an eye out for the next intake coming to Bastow in 2020. We spoke to Kathy about data, collaboration and equity in education.

So, Kathy, you’ve written extensively about data and the important role it plays in learning and teaching. Why does it matter? Why is it so important?

KATHRYN PARKER BOUDETT:

Um, I feel like data gives us a window into what’s working, what students are learning, what they’re not learning and it allows us to ask really important questions about our own practice and so being able to look at evidence of student learning and here I mean broadly defined evidence, everything from a standardised state test all the way down to an exit ticket. I don’t know if you guys have that expression but like something that you would have a student answer as they’re walking out the door of a class that just gives you a quick read. Did they get it? Do they know what I wanted them to know at the end of this class? That can just provide really powerful guidance to teachers so you’re not shooting in the dark.

So, around the globe in every industry, people are getting more and more savvy about how to use information for decision making, how to present information in ways that people can quickly see patterns and understand when action might be needed, potentially what kind of action. And in education, we should accept nothing less. And I’m not saying we need to have like a tickertape of information going by, but we need to embrace the extent to which evidence can help us do our jobs better.

ANGELA:

So, there are great benefits in using data properly for school improvement. What are some of the risks or the potential traps?

KATHY:

So, gosh! There’s so many! (laughs) I just…I feel like data… I gotta call it ‘dar-tar’ with you, right? Data, it can be used as a weapon as easily as a tool and I think the challenges come when people assume the data means more than it… You know, I don’t know if I’m going to be able to say ‘dar-tar’ so I’m just going to say data. (laughs)

ANGELA:

Absolutely.

KATHY:

When people read too much into the data and they think that, “OK, this is truth,” and not understanding sort of the assessment literacy that’s needed to appreciate what kind of conclusions, what kind of inferences can be drawn from the data, so I feel like that’s one concern.

It also can be used to sort of reinforce stereotypes about particular subgroups. If you sort of look at comparisons, people may use it just to confirm, “That’s what I would’ve expected from that particular group,” and not really sort of dig in and understand what’s under the numbers. So, I think that’s a piece of it.

People can use data just to blame. They can use it to point fingers and say, you know, “Well, it’s that teacher’s fault for a particular set of outcomes.” Or it’s that kid’s fault or it’s their family’s fault. I mean, there’s just a lot of ways in which it can be used negatively but there’s so many positives too! I hope you’ll ask about those! (laughs)

ANGELA:

Yeah, so what are some of the positives of using data?

KATHY:

I mean, I think the best part is when you use data to get a common understanding of what students know and what teachers know about sort of their own students, about their craft and we use that to ground a conversation. Because so often, teaching happens in silos. I don’t know if that’s as much true here in Australia as it is back home but where you go into a classroom, you close your door, you do your very best, most teachers, for their children. But to have a grounded conversation with your colleagues where you can talk about what you’re seeing, in terms of student learning, and have a real back-and-forth about what does that mean about your teaching. I feel like that’s really powerful. It can transform what it means to be a teacher from just sort of a solo kind of thing to a very collaborative enterprise where you personally are trying to get better, but you do all the time and you’ve got some data to check and see how you’re doing with that.

ANGELA:

It’s interesting that you talk about collaboration because that obviously is a really important starting point. How do we do that well? How do we, as a sort of leadership team in a school, how do we kick off that collaboration process?

KATHY:
Hmm. I’ve done a lot of thinking about that because it is so important. And I think the most important thing is to develop a sense of trust among people on the team.  And that sounds like a high-falutin’ thing. Like, you know, “To do this for today is to establish trust with my colleagues!” And so, we’ve thought of ways that you can kind of break down that sort of…really, it can be a long-term goal, into doing some more intermediate things that can sort of work toward that. And so, I’m thinking about everything from just establishing sort of norms of interaction, actually agreeing on how are we going to work together, what are sort of our standards for whether we’re going to show up on time to a meeting, whether we’re going to stay on-topic when we’ve agreed to something.

I guess, my favourite norm is sort of assuming positive intentions just as we’re going to start out that we all agree, or we all believe that we mean the best for children and take that as a starting place. And we’ve actually had to add to that norm ‘and take responsibility for impact’. Because sometimes, if you assume positive intentions of somebody but they say something that’s sort of hurts or that sort of gets you wrong, you need to be able to then tell the person how whatever they said was received and they need to take responsibility for, like, “Oh, I respect that was your experience of what I said,” and have that back-and-forth. It gets down to kind of helping people not avoid but be able to sometimes lean into conflict or if not open conflict, disagreement and for that, having a norm around taking an increased stance and asking a question before you assume you understand what the person’s trying to get at, like, “Tell me more about what you mean by that.” Or a norm around grounding statements and evidence, like, “What do you see that makes you say what you just said?” That can be a really powerful thing. So, norms, you know, it’s the kind of things you do the first day, ideally, of working with a group. And then you gotta think about how are we going to keep those norms alive so we get to the trust, which is sort of the end goal where we can push each other.

ANGELA:

So those norms sound great. They sound really helpful. How do you keep them alive beyond the first interaction?

KATHY:

Um, you gotta keep coming back to them. You have to, um… I mean, so we do simple things, like put them on the walls or when we’re running an institute, we’ll have a norms tent with the norms listed so they’re kind of in people’s vision. But much more important is having conversations about how are we doing on those norms and creating a safe enough space that people can acknowledge when maybe, “We’re not sure that we’re really sticking to evidence here,” and, you know, talking about that.

So one of my favourite things is just that norms tent that’s on the table, having people at the beginning of a meeting sign their initials to a norm that they would like the group to help them work on and that can be a really positive way of saying, you know, “I’m inviting the group to support me in this.” And then, if you can go the extra step and say, “What would help you?” So, the one that comes up so much is ‘be here now’. So that’s a norm that we love in Data Wise. But in this world of hand-held phones, where you can get instant information about what’s happening on the other side of the world, sometimes somebody will say, “I have trouble being here now, but it would help me if we all agreed to put our cell phones in the centre of the table for this meeting.” Or, “It would help me, if you see me sort of glancing at my email, if you kind of just give me a little elbow.” To be authorised to support somebody like that in collaborating can be really liberating actually.

ANGELA:

Thank you. So many of our listeners will know a lot about Data Wise but for those of us who are less familiar, can you briefly explain what underpins the methodology?

KATHY:

Sure. So, I’d say at its heart, Data Wise is helping people to learn to work together to seek the truth through evidence and I define ‘evidence’ very broadly here. So, I don’t want listeners thinking, you know, “She’s talking about state test scores,” or something like that. We’re talking about a broad range of evidence, you know, student work, conversations with students, projects, oral reports, all of that, using all those things together to figure out how to do the right thing by children, how to hold everybody – and we talk a lot about ‘all students’ – holding all students to high expectations where they can learn and acting and adjusting on our own instruction to give each student what he or she needs.

ANGELA:

So, what you’re encouraging us to do is to really think very broadly about data, the data we tap into, the data we use and how we think about it and think beyond where we would normally go for that data?

KATHY:

Absolutely. Data Wise was born out of sort of a sudden preponderance of data that was available through our ‘No Child Left Behind Act’ and what happened was that people were very narrowly defining data to mean whatever the score was on the state test that was being used to stay in compliance with that Act, that was the be-all-end-all. And we felt there was some tremendous dangers in doing that because, you know, a state annual assessment is going to tell you…it’s not going to really actually going to tell you much of anything about what an individual kid needs instructionally in order to learn. It will help you figure out what questions you might want to ask, what data you might want to dig into more deeply to understand. And so, with Data Wise, we say if you’re going to start with sort of high-level, high-stakes, standardised-type tests, that needs to be a launching point for a more in-depth inquiry into what’s actually being produced.

And I just love the data that is something written in a student’s own hand so, you know, student writing, students’ sort of drawing, projects. I love being sort of a fly on a wall watching a group of students having a conversation about an assignment and that’s data. I mean, who is understanding what’s being asked, who’s pushing back, who realises the assignment itself is confusing and, I mean, that helps you get a window into student thinking and that’s what we’re shooting for.

ANGELA:

So, Kathy, we’ve talked a lot about student data but what about data around teaching?

KATHY:

So critical. I mean, I really think of student data and teaching data as two sides of the same coin and sort of a great failure would be to go straight from looking at student performance data to, boom, “What does that mean for our action plan?” Because you want to look at, “What does our teaching practice look like?” And this can be… This can feel like a very personal thing, to allow people to come into your classroom. And here, I’m talking about teachers watching teachers but still, that can feel like, an invasion or like you don’t feel safe, like you’re trying to put on some kind of a show. But the ideal is when teachers can develop an ongoing practice of getting into one another’s classrooms looking for evidence of what kind of instructional practices are being used, how are those being received by the children and then, stepping back with colleagues and looking for patterns.

So, if we either physically visited one another’s classrooms over a period of time or even just watch videos of one another’s practice, what are the patterns we’re seeing in how we interact with children? A pattern that often gets discovered that you don’t even know you’re doing it until you have the ability to step back and look at practice is teachers doing too much of the talking, teachers answering their own questions and it’s not what they mean to do. They think that they’re providing opportunities for students to really, you know, sort of struggle through a piece of analysis but when they watch a video of themselves or of their colleagues teaching, they’ll say, “Wow. We’re jumping in so quickly.

So when teachers have the luxury to, in a safe environment, look honestly at what’s happening in practice, they’re able to observe patterns that they might not have otherwise been aware existed but that they know need to be addressed in order for students to have the opportunities to do the kind of deep thinking that will allow for the kind of achievement that they want for kids.

ANGELA:

So, I’m sure our listeners will be very interested to understand how we might measure improvement across a system. I’d love to hear your thoughts on how we might do that.

KATHY:

Mm. So, I mean…so we like to think about a system, ideally, as modelling the kind of commitment to continuous improvement that they expect of schools and so that requires that the system-level leaders are willing to engage in inquiry themselves and to look at their learners. So, when we’re talking about a school context, it’s pretty clear that the learners are the students sitting there in their seats and getting the instruction. But when you’re at the system level, your learners are often principals or teachers. So, if we think about principals potentially as being the learners, then the system-level folks have an obligation to the development of those principals that is similar to what a teacher might have to their students.

And so, people can look at the sort of academic outcomes of a school and maybe look at sort of trends over time and how that school is performing against its previous performance or how does it perform compared to similar schools in the area and trying to figure out, from there, what’s the next level of work for the school. It can be maybe particular content areas are in need of improvement or there are some groups within the school that are falling behind in one school and not so much in the other. But the key and what would make it ‘data wise’ would be if it’s not just a matter of seeing a disparity and then blaming the principals or the teachers within the schools for what they see but it’s like, what are WE doing, as a system, that is creating the outcomes that we’re seeing? How are we supporting the principal or the teachers? What are we putting into place that’s allowing them to deliver the kind of education that children need?

And when that happens and there’s a humility that happens when system-level leaders accept responsibility for the role that they play that, I think, can be incredibly empowering for schools if they know that that’s what’s happening. So, in a recent release of our Massive Open Online Course, we had to add this whole section which is ‘What does it look like for a system to engage in Data Wise itself?’ And hearing from the teachers that, “I feel OK at this continuous hammering away, trying to get better each day, knowing the people at the system level are doing the same thing and they’re struggling, they’re trying something and maybe it doesn’t turn out exactly the way they want it to and they’ve got to make an adjustment. And so, me as a teacher, then I feel like, well, that’s what I do too.”

ANGELA:

So, it sounds like it’s everyone at every level taking responsibility and then supporting each other.

KATHY:

Yes. Yeah. And there can be a sort of ‘we’re all in this together’ kind of moment, where a principal might be talking to a supervisor and saying, “We thought we had a handle on what the learner centre problem really was and then realised, when we got more data, that actually, there was a fundamental issue we had to address first.” And, you know, a supervisor might laugh and be like, “That reminds me of when we were doing our previous inquiry cycle, we ran into something like that.” And it’s just like this is the way we work. This is what it means to be a professional in education and I think that makes everybody feel like they’re part of a shared enterprise.

ANGELA:

So, leadership’s an essential element in the way we think about systems and networks. Who inspires you as a leader?

KATHY:

Hmm… I would say I’m most inspired by teachers because I feel like they’re the ones who are right there in that magical moment where students are learning and growing and sort of expanding their potential. And just, when I get to talk to a teacher who’s so determined to open doors for their students, it gives me the shivers. And being able to amplify the voices of teachers. So, Data Wise was very much created out of conversation with principals and teachers about what are they doing day-to-day that is allowing them to harness data for good. And when I get to sort of talk with teachers about how they’re doing that, it makes we want to keep doing the work that I do.

ANGELA:

Equity is clearly something that matters to you.  What is the link between data and equity?

KATHY:

Oh, I love that question. I feel like that’s just, um…that is my personal question. I feel like the why that wakes me up, the reason why I want to sort of work hard and figure things out is because I believe every single kid deserves a really high-quality education. I feel like the current system within my state, my country, our world isn’t set up for everyone to have equal opportunity. And so I feel like we need to be able to shine a light on the places where inequities exist and data can be very powerful for that, in terms of giving people some kind of incontrovertible evidence that either certain subgroups are underperforming or children with disabilities are not able to achieve the levels that we know they could or should be able to. And so being able to point to evidence allows for a very grounded conversation in what needs to get better.

ANGELA:

So, this work very much aligns with your purpose, your sense of purpose?

KATHY:

Mm-hm. Yeah. And, I mean, I find more and more, as we work with schools, we realise that you don’t want to lead with, “And now, we’re going to bring in a data inquiry process.” Because nobody is inspired by a data inquiry process in and of itself, right? But if you can get people to think about what are each person’s individual ‘why’ and what’s our shared ‘why’ as a school or as a system, what brought us all into education in the first place, I bet it wasn’t data. I bet it was because we believed that education can be a really powerful lever for increasing equity in this world, for developing students that are capable of solving the problems, that will give us a more just and liveable planet. And if you can tap into that ‘why’, then data becomes a tool. It’s a tool for serving a purpose as opposed to an end of itself.

ANGELA:

I’m sure that there are lots of people who are listening who love data, who are saying, “That’s why it matters! That’s what I’ve been trying to say.” And you’ve said it so eloquently.

If you’re a leader in a school or a network today, where would you focus your energies?

KATHY:

I mean, I think building a team that is focused on student learning, um, is critical, taking the time to build a sense of shared purpose and showing people that what you’re looking for is long-term, massive…I hear that word so much here in Australia! But we’re looking for substantial, deep instructional changes as opposed to the much more standard, “Oh, I looked at the data. It seemed like people were scoring poorly on these four questions so let’s re-teach the content of those four questions.” And that’s…that is not what’s going to motivate people to want to be sort of all-in. So, creating a sense of shared purpose that this is about making fundamental changes in the relationship between students and teachers and the content that they teach, that magical instructional core that is so much in the vernacular, helping people believe that that’s what we’re about and then creating the structures around the team that allow it to be successful. 

I mean, I talked earlier about the power of norms. I also believe very strongly that thinking about the way meeting time is used is absolutely critical for leaders. So, if you’re a leader and you feel like, “Gosh, I’ve got so many meetings and it just seems like it’s such a waste of time. It’s keeping me from  the real work,” well, look closely at those meetings and if you have control over how they’re run, how clear are the objectives for a meeting, have we set up rules so that it’s not always the leader who is facilitating a meeting but maybe that role is being rotated and we’re getting multiple perspectives on how to run an effective meeting and multiple voices in on the conversation, I feel like that can be a really powerful lever.

I mean, one thing we find sometimes, ‘cause we have the Data Wise book which talks about the eight-step improvement process. But soon after we wrote that, we realised we needed a whole book just about how to run a good meeting. Because what happens is if the work of Data Wise takes place in meetings and those meetings aren’t being effective, you’re not going to get the results. And so, we often have people say, “Let’s start with thinking about how we use our collaborative time developing a whole new way of doing business around treating those meetings as places where learning happens, not just like information getting delivered to people but actually maybe even bringing in some of the best practices we know about adult learning, or anybody learning, is giving people rich opportunities to test out their thinking, giving multiple people voice. It’s a very powerful lever for changing the culture.

ANGELA:

I’ve heard that you apply all of this thinking in your own context, in the work that you do.

KATHY:

(laughs) Who told you that?!

ANGELA:

Through the grapevine. Are there learnings along the way for you, around what you’ve tried and what’s worked and how to bring some of this to bear week by week, day by day?

KATHY:

Um, yeah, I mean we feel like to be credible we need to practise what we preach. That was, I think, partly how we got started. But I find that I still think meetings are hard.

And so, I would say that it’s doing what we call sort of a ‘plus/delta protocol’ at the end of every meeting that has allowed me to keep my sense of humour around meetings and to not expect that they have to go perfectly every time. And so, um you ask everyone at the meeting to say what’s a plus, what’s something that worked really well in the way we interacted together during this meeting and then, what’s a delta, so that’s the Greek symbol for change - what would you have changed about this meeting to make it even better? And we find if we can model, often at the beginning of a meeting, if I’m working with a new group, like what’s a nice, juicy delta?

So, a delta like, “I wish we had coffee,” that’s not a… Yeah, that’d be a nice change and I actually then do provide coffee the next time, to show, “Look. I’m going to pay attention to these deltas. We’re going to do something different.” But, you know, a delta about, “Well, I noticed that only about half of us spoke up in this meeting and I’d love to hear from more people.” Then, you can enter your next meeting saying, “OK, so a delta coming out of our last meeting is we didn’t hear from everybody and we’re going to put into place the following things that will allow for more equity of voice,” something like that, so that the meeting itself is a co-created enterprise where we’re all taking responsibility for being effective in working together and learning together.

ANGELA:

So, the people listening in could use that plus/delta.

KATHY:

Right there. Yeah. And I didn’t make it up. I’m not even sure who did. It was just sort of in the water. But I would say, for listeners, it could be the simplest yet single biggest thing you could do to get yourself on a path toward continuous improvement, would be to have conversations about the process that you as a team are using and be open to changing it. And when people see that you made a change as a result of a delta that was offered in a meeting, you’d be amazed at how much better the deltas get the next time ‘cause they’re like, “Wow. Alright. Well, this is not an exercise we’re doing. This is a practice. This is something that we’re committing to.” So, I think it’s as close to magic as I’ve found in this world.

ANGELA:

Kathy, it’s been an absolute pleasure speaking to you today.

KATHY:

Well, thank you so much for having me here. I just feel like this work is so close to my heart and the opportunity to talk with your listeners about something that matters to all kids has been wonderful. Thanks.

ANGELA:

Thank you.

Thank you for listening to the Bastow Educational Leadership podcast. If you’ve enjoyed this conversation, why not tell your friends and colleagues. And join us next time! You’ll find episodes on the Bastow website, and you can listen or subscribe wherever you find your podcasts.