In a NYT article about Bill Gates’ venture into teacher evaluation, I’m stuck on this sentence:
Researchers and educators involved in the project described it as maddeningly complex in its effort to separate the attributes of good teaching from the idiosyncrasies of individual teachers.
Why Gates investing $335 million in trying to separate those two things? Because he wants to find the generalizable lessons of good teaching. He tells the Times:
What’s unbelievable is how little the exemplars have been studied. And then saying, ‘O.K., How do you take a math teacher who’s in the third quartile and teach them how to get kids interested — get the kid who’s smart to pay attention, a kid who’s behind to pay attention?’ Teaching a teacher to do that — you have to follow the exemplars.”
The assumption is that the ‘exemplars’ do something other teachers don’t do but can learn how to do.
It’s the same thinking behind the UN Peacebuilding Commission, which I spent my summer following around. The operating philosophy there is, “Let’s add ingredients we think are often missing after conflict — money and attention — and test theories we have about what helps keep conflict from coming back, so that we can apply the successful ones in other countries.”
I looked at rather a lot of PBC programs, on paper and in the field. The ones that worked were unreplicable. Or at least, the only thing replicable was, “Know the actors, context and history, and do something that seems important in those circumstances.”
Why not give a little stage time to the idea that success may be about idiosyncrasy? That success is not a generalizable set of actions we can study, in order to build a universalizing theory about “this thing that works,” in order to replicate that thing elsewhere? It would require humbler –more realistic? — expectations of what money does, but maybe it would help us use the money better.
Nice post. I agree and think people get in trouble when they think they’ve found a cookie-cutter solution for something. It’s something I hope to write more about. You’re right, I think– idiosyncrasy, or taking idiosyncrasies and context into account, is a huge component of “success” (however you “measure” that).
Thanks, Carol. (Uh-oh! The “m” word!!)
Jina, you raise such a great point. A few half-formed thoughts on the topic:
I agree there’s a general tendency to use this “best practice” approach across issues (education, peacebuilding, etc). It’s how many of us think about solving problems. The whole management consultant industry seems to be built on this premise. It’s actually quite useful in the sense that it gives us *something to do* — even if that *something* doesn’t result in any improvement.
To overcome that thinking, we need to offer a new analytical and operational framework. You identified the analytical framework with “know the actors, context and history” etc., and there’s even an epistemic community around that. But people will stick to the “find and disseminate best practices” model until we can build an operational framework to match the analytical one. That requires a funding community and actual organizations that can operationalize the principles. I don’t think we’re there yet.
Jina, this is really interesting!! A totally valid point I haven’t seen raised elsewhere. From a teacher point of view, I think the most important thing is paying attention every single nanosecond to whether or not a student is understanding what you’re doing and adjusting accordingly. Instead of waiting for “metrics” in the form of an exam, evaluation, assessment, you’re getting data every single second from the student so you can deliver a totally individualized experience. How could this apply to the PBC world?
In that case, would the “best practice” be to immerse yourself in the idiosycrasy, listen like crazy, and adjust accordingly? Is the “replicable” solution the listening and attention in order to create a hand-tailored solution to every different case?
Thanks for sharing, Rebecca. Never thought of comparing this to education. I guess there’s an obvious time difference here — teachers spend day-in, day-out with students. Development workers largely don’t. And I think a lot of people think they are creating hand-tailored solutions to the cases they see… but inevitably bureaucratic culture and interests force them to adjust.