Today we have the first NeuroNeurotic guest post! The following short rant was written by my colleague Lee de Wit. In this he talks about the recently published “Teaching Excellence Framework” in which UK universities are ranked based on the quality of their undergraduate teaching… If you are also a psychologist and would like to write and sign a more formal letter/editorial to the Times Higher Education outlining these points, email firstname.lastname@example.org
As psychologists when we create a new way of measuring something complex (like a personality trait) we have to go to rigorous lengths to demonstrate that the measures we use are valid, reliable and that we classify people meaningfully.
When it comes to measuring teaching in higher education however, it seems we can just lower the standards. Apparently the TEF is meant to help students make meaningful choices, yet I can see no evidence that it is a valid measure, no evidence it is reliable, and no evidence that it meaningfully clusters Universities.
Validity – One of the key measures used in TEF are student satisfaction scores – yet we already know that they are not a valid measure of teaching quality. In fact there are meta-analyses demonstrating that high student satisfaction scores don’t even correlate with learning outcomes.
Reliability – Apparently it is fine to just have a panel of 27 people make some subjective judgements about the quantitative data, to classify Universities. No need to have two panels rate them and then check they come to similar judgements.
Clustering – in terms of the underlying distribution of the data, no need to seriously think about whether there are meaningful clusters or more continuous variability. Gold, Sliver and Bronze – job done.
If there are any academics tweeting today about your University’s strong result, I would seriously call into question the excellence with which you can teach critical thinking to your students.
The one lesson I would take from this for UK Universities, is that we are clearly failing to educate politicians and policy makers to think carefully about evidence based policy. Presumably most of the key players in designing and implementing TEF went to UK Universities. So I’m worried about what they learnt that made them think this was a good idea.