Running Experiments

Transcript

The number one challenge I see in leaders who have a really strong aspiration to make things better is that they have a tendency to move towards trying to roll out their idea across a whole organisation, a whole school, a whole team, as the first stage of improvement. And I get where this comes from. I mean, I respect where it comes from. It comes from a place where you think, “You know what? I want to make things better. I think I’ve got an idea here that could make things better. I’ve done all this thinking, now let’s have a launch day and let’s roll it out and hope it works.” Maybe they borrow from some thinking from that change guru John Cotter, and they say, “I’m going to get some buy-in.” That is, I’m going to get people’s cognitive and emotional commitment and then I’m going to roll it out with everyone.

But if we had to learn lessons from organisational change, if we had to learn lessons from implementation science, one of the things we just have to be aware of is that if you’re working in schools and school systems, you are working in complexity. And that complexity, it’s not something you solve and work through by just planning more, writing a more detailed plan and then implementing with fidelity. No, actually, working in complex human environments leads us to a more humble stance. We have to say, “No matter how good my idea is, no matter how grounded it is in the best available evidence, this may not work here.” And it’s in that liberating insight for me, that combination of complexity and the acknowledgement that in that complexity, there’s a whole range of things that I don’t even know yet that might go wrong. It liberates us to release this notion of trying to go to everyone at the beginning and embrace the art and science of running small experiments.

Running experiments is one of the great ways to accelerate learning in educational change. You see, what we want to do is to de-risk the original contact between our conceptual ideas and the day-to-day reality of the schools we’re trying to improve. And so, as a rule of thumb, if you want to do something at a whole school, you should always drop down at least two levels. Which means don’t work with a whole subsection of a school, probably just work with one team. One team of teachers or one cluster of parents or one subgroup of students. It depends of course on the target that you’re working with. You might say, “well, no, I don’t want to just work with that one team. I want to get to everyone.” Yeah, the fastest way to get to everyone in a way that’ll work is to get to a few. And so, start to think about an experimental lens to your work.

Now, I’m the former high school science teacher, so it should get no surprise here that I’m interested in experiments, in testing hypotheses. But the idea is that you de-risk this work. You’re not going to waste a lot of people’s time, and you’re not going to waste a lot of your change reserves and energy. You’re just going to get moving. I often say to people, “Start small, move fast, and learn.” Because what you’re doing is that you can start small, so you’ve only got a few. Maybe it’s a small number of people, small number of teams, or maybe you’re experimenting by working with a larger number of people, but you’re only taking a very small amount of time. For example, you’ve got a new idea for professional learning. Don’t roll it out for everyone at scale, but maybe you just take 20 minutes of an upcoming full day professional learning session.

So, you can limit your experiment by the number of people. You can limit your experiment by the amount of time. And you’re just holding things lightly, because you’re trying to really start small and move fast and to learn well. Because we’re trying to just get into the work, and we know that there’s lots of ways that this could go wrong, and the number one thing we’re trying to do is to set ourselves up for learning. You see, if you go too quickly at a large scale, you actually can’t learn very well. There’s so many things operating at the same time. The scientist in me says there’s so many variables that are moving at the same time. It’s very, very hard to isolate what’s not working and what is working and draw out lessons from that. And indeed, you’re probably going to find yourself in a situation where you’re dealing with a lot of the cynics and the sceptics and some of the pushback, and you’re actually not focused enough on improving your solution, and you end up too focused on some of the change dynamics.

But if you can liberate yourself to say, “Hey, what I always do when we get moving with a new idea or refining an old idea, we always run little experiments.” And I think, “How quickly can I get into action here? Can I get working with a small team, a small number of volunteers? Can I get working for one month or one term to start to develop some ideas, refine our work?” And what you’re going to do if you work with a few, and wherever possible I say, “In the early phases, work with people who are high-skill, high-will, those who are willing to give it a good go.” And you might say back to me, “Well look, Simon, just because it works with people who are high-skill and high-will, it doesn’t mean it’s going to work with everyone else.” I say, “Yeah, good point.” But if it doesn’t work with those who are high-skill and high-will, there’s no way it’s going to work with everyone else.

So in your early quick tests, often working with people who are pretty high-capability and they’re pretty motivated is a great way just to do a quick rigour test. Do we even have the right pieces here? Have we got the right thinking in places? Is there any big bits that are missing? Later on you might say, well look, I want to bring in some people who are what Adam Grant calls positive disagreers. It’s this idea of bringing in some people who might be a little bit of a sceptic. I’m not talking about bringing in the cynics. I’m talking about bringing in some interesting sceptics, some people who might have some real standing in the common room. And you’re not asking them to be your new recruits. You’re not asking them to go out and be your new champions. You’re bringing them in early because, actually, even though they might give a little bit of pushback and ask some difficult questions, they’re exactly the kind of conversations you need to have early on in the work, because they’re going to help you refine it and simplify it and get to something that might have a higher likelihood of working.

So my question at the moment is, where are you going too big, too early? Do you have a tendency to stay in your own thoughts and planning and re-planning and analysing and getting it all right with a plan to, at some point at the start of a term or a start of the year, launch it with everyone and hope it works? Might there be an opportunity here to just acknowledge the complexity of the environments in which you work? Which is liberating, by the way, because no matter how smart you are, no matter how well-read you are, you can never know everything you need to know at the start of a journey. You’ve got to get into your unique context. And how could you run some small experiments, get moving over a couple of weeks with a few people, get the feedback loop to hear the things you probably don’t want to hear but you need to hear, about things that we haven’t thought about around the time that’s needed and the resources that is needed?

Sometimes you might put something in front of people and you think, “Oh, this is some of my best work. It’s so clear.” And they really don’t get it, or they totally miss the message. And I want to acknowledge, it can be frustrating. But how much better that you’ve done that with a few in a low risk environment? You can make that adaptation and adjustment, and make sure that when you’re moving to a larger scale launch of some sort, you’ve actually already learnt a lot of lessons cheaply. And most importantly, through those experiments, you’ve got some social proof. That is, you’ve got real examples and voices that can speak to the work that they’ve trialled, and bring those voices as you move from experimentation to expanding.

So is it time for you to learn the art and science of running educational change experiments? Experimenting is the best way to acknowledge and respond to the complexity of school environments. It’s also a core approach to accelerating your own learning for improvement, as you and a small group of others in a design team start testing your hypotheses out in the real world, collecting some real data and some honest feedback from the people who are involved in the early change, and use those experiments in the next phase of the work. So is it time to start bringing experiments into your school change approach? Are you going too big and therefore learning too slow? Is it time to run fast quick experiments? Course correct, adjust, learn your lessons in a low risk way. And once you’ve cracked the model for improvement, then start to take it to others. It’s time to harness the power of small experiments.

Well, thanks for joining me. I hope you’re getting a huge amount of value out of these ideas. One last request before you go. I’d genuinely appreciate it if you could subscribe, rate, and review this show. It’s one of the easiest ways for us to get these ideas into the hands of even more educational leaders.