From the archive: Stop repeating nonsense about ‘bad’ teachers

Jack Marwood /   November 10, 2014 at 8:39 PM 3,206 views


There is an incredibly irritating ‘Bad teacher’ factoid which keeps cropping up in education. A factoid is not a fact. It is ‘an item of unreliable information that is reported and repeated so often that it becomes accepted as fact.’ The factoid to which I am referring is roughly as follows:

Teachers who work in a given school, and therefore teach students with similar demographic characteristics, can be responsible for increases in math and reading levels that range from a low of one-half year to a high of one and a half years of learning each academic year.

A good teacher can get 1.5 years of learning growth; a bad teacher gets half a year of learning growth.

You can find examples of this factoid everywhere, but the thing which has finally made me write this was seeing a head teacher – someone who, I have no doubt, has the best of intentions – making statements based on the factoid:

Headteacher Lynne Gavin, of Pakeman Primary School, says (between 0:20 and 0:36), “We looked at the Sutton Trust and we came across a piece of research which showed the impact of poor teaching on an average child and the devastating impact poor teaching can have on a disadvantaged child.” This is based on a factoid which is simply not true. Here’s why.

The ‘Bad Teacher’ factoid first appeared in 1992, in a paper by Erik Hanushek, who is responsible for the two quotes above.  The paper is actually about ‘The Trade-off between Child Quality and Quantity’ – whether the number of children in a family has any relationship to the ‘performance’ of a given child. The data used came from a sample of ‘low income black students from Gary, Indiana’ in 1972-1975 which, the author notes, ‘is clearly not representative of the entire population.’

In his conclusion, Hanushek blithely states that, ‘there is no doubt that teachers vary dramatically in effectiveness. The difference in student performance in a single academic year form having a good as opposed to a bad teacher is can be more than one full year of standardised achievement.

This damaging nonsense is a classic error of the Value Added delusion, which defines a ‘good’ teacher (or school) as one with ‘good’ results and a ‘bad’ teacher (or school) as one with ‘bad’ results. Additionally, the Value Added delusion ignores the crucial fact that a closed system ensures that it is impossible for all teachers or schools to ‘add value’ – it’s essentially a zero sum game. Furthermore, the average ‘student performance’ of different cohorts will naturally vary considerably because of underlying distribution of test scores, with some gaining ‘more’ and some gaining ‘less’. Finally, the delusion cannot separate correlation and causation, and simply uses Halo Effects to attribute the features of ‘success’ or ‘failure’ to ‘good’ and ‘bad’ based on badly-measured outcomes which can only be allocated from a limited – and closed – range of possible outcomes.

There is a great deal to object to in the Hanushek study, not least that the number of observations is tiny, the range of data is limited (and forty years out of date), the findings cannot be generalised and the underlying assumptions are all contested by other educational academics (see How unstable are ‘School effects’ assessed by a Value Added Technique? by Stephen Gorard, Rita Hordosy and Nadia Siddiqui, 2011), by Mathematicians (see Mathematical Intimidation: Driven by the Data by John Ewing in the May 2011 Journal of the American Mathematical Society) and by the American Statistical Society (in its statement on Using Value-Added Models for Educational Assessment, 2014). If the findings were robust, other studies would have corroborated them, but this hasn’t happened in the 22 years since Hanushek’s study was published.

Hanushek works in the USA, but his incorrect claim has been taken up by those in England who want to believe that ‘bad teachers’ negatively impact on a pupil’s ‘performance’. The worst offenders are the Sutton Trust, who are quoted repeatedly by policy makers and politicians, and cited by the head teacher above, who appears to be referring to the Sutton Trust’s 2013 report ‘Testing Teachers’ which references Hanushek on page 7.

An earlier report by the Sutton Trust, ‘Improving the impact of teachers on pupil achievement in the UK – interim  finding’ actually had Hanushek as an advisor. This report repeated the factoid (Improving the impact of teachers on pupil achievement in the UK – interim findings, page 2).

The evidence for this is – guess what – Hanushek’s own factoid (Improving the impact of teachers on pupil achievement in the UK – interim findings, page 5).

Whilst the Sutton Trust has become a trusted source of advice for many in teaching, it is worth remembering that it is a non-government organisation with its own distinct agenda.

The factoid is frequently used in educational research. The Institute of Education’s recent ‘Primary and secondary education and poverty review’ uses it to make assumptions about ‘Teacher Quality’. The Centre for Market and Public Organisation at Bristol University’s report ‘Do teachers matter? Measuring the variation in teacher effectiveness in England’ leads back to the factoid too.

Politicians often quote the factoid as fact. The Social Mobility and Child Poverty Commission’s report Cracking the Code, quotes the Sutton Trust’s ‘Improving the impact of teachers on pupil achievement in the UK’, which uses Hanushek’s factoid as detailed above.

There are countless other reports which all lead back to Hanushek:

The ridiculous thing about all this is that even Hanushek knows it’s nonsense. He’s been trying to find evidence for his factoid since the late 1960s. His research over the past 45 years has not managed to prove his hypothesis, and he can find no link between anything which would identify an effective teacher other than the defined/circular ‘good test scores = good teacher’ delusion:

Observed teacher characteristics do not represent teacher quality. The consistent finding over four decades has been that the most commonly used indicators of quality differences are not closely related to achievement gain, leading some to question whether teacher quality really matters (see the review in Eric A. Hanushek and Steven G. Rivkin 2006).” (Generalizations about Using Value-Added Measures of Teacher QualityHanushek & Rivkin, 2010)

Essentially, the factoid – deliberately or otherwise – mistakes correlation for causation, and no-one has yet found any way whatsoever of identifying any underlying factor which links those who are identified as ‘good’ or ‘bad’ teachers by delusional Value Added number crunching. Some classes may make a year more progress on average than others, but this has nothing whatsoever to do with any underlying common teacher characteristics.

There’s gold in them thar factoids

So why on earth has this factoid been repeated over and over again? The answer, as with many things, is to follow the money. Those who want to believe that teachers and schools can be divided into Good/Bad are usually selling something, trying to get money from philanthropists, businesses or governments, or simply justifying their own existence.

The Gates Foundation is a big believer in Hanushek’s factoid, even though their multi-million dollar Measures of Effective Teacher project was a monumental waste of money, misinterpreted its own data and couldn’t find any way to identify an effective teacher. And what they came up with would cost $5 billion a year in the USA alone. That’s why people want to believe the factoid.

In this country there are any number of people and organisations trying to make a living by selling assessments of ‘student performance’, teaching training and continuing professional development, ‘progress tracking’ systems and all manner or other gubbins. Ofsted survives by attributing Halo Effects to delusional Value Added calculations. Many of these people are well served by a narrative of Good/Bad teachers and schools. But we need to stop believing things which are simply not true.

Teachers have succeeded in driving the charlatans behind Brain Gym and Learning Styles out of the mainstream discourse about education. Despite thousands of people being hoodwinked by the delusion of observing pupil progress in real time, we have stopped believing this to be true, by showing the underlying assumptions on which the delusion was based to be false. Even the drifting tanker that is Ofsted has stopped writing handbooks in which a preferred teaching style was proscribed despite ample evidence of its efficacy. We have accepted the errors which have been made and moved on.

Just Stop it

I work with primary school children. Being young, many mess about a bit in class, or attempt to when they think I’m not aware of what they are doing. Nearly every time I see a child who is messing about, I have a small voice in my head which is yelling, “Just stop it. Now. Just. Stop it.”

I feel the same about those who believe the Hanushek ‘Bad teacher‘ factoid. You have been had, either by choice or your own gullibility, or because no one has pointed out that it simply is not true. As I keep saying, teachers and schools need support, not condemnation for factors outside of their control. In this case, teachers should not have to bear the brunt of bad ideas based on something which is simply not true.

By all means aim to help teachers to get better at what they do. But stop believing that some people are ‘good’ and some people are ‘bad’. Stop repeating nonsense about ‘bad teachers’, and saying that ‘good teachers’ are responsible for higher ‘learning growth’. Education is more complicated than that, and you need to stop believing things which are not true. Just stop it. Now. Just. Stop it.

Courtesy of Jack Marwood at Icing on the Cake

Sign-up for the Guerilla Daily newsletter – the best frontline and independent blogs every week day.

Please comment with your real name using good manners.

Leave a Reply