Your own personal PISA - what does the TALIS show us?
Every five years, the Organisation for Economic Cooperation and Development (OECD) conducts the Teaching and Learning Survey (TALIS) in which they ask lower secondary teachers from around the world about their working conditions.
The most recent survey, TALIS 2013, sparked some comments in the press and by politicians when the results were released in mid 2014. For instance, responding in the The Telegraph to the finding that 63 per cent of teachers in England frequently give different work to pupils in the same class, then schools minister, Liz Truss commented, "In the best rival systems, only 25 per cent of teachers do this, making classes simpler to teach and making better use of great teachers' time."
This seemed pretty interesting to me because the TALIS data is available online. Given that the OECD's latest achievement data on fifteen-year-olds from the Programme of International Student Assessment (PISA) is also available online, I thought it might be a bit of fun to plot a graph and see if there was a correlation between the two sets of scores. As a maths teacher, I decided to compare the maths PISA score with the frequency of differentiation.
Caution is needed here because differentiation and differentiated instruction mean many things to many people. For instance, differentiation could simply mean targeting teacher questions to specific children. In some versions of differentiation, students may be given different tasks due to some conception of their learning styles; a popular notion with little evidence to support it. However, the data from the survey relates to the specific practice of giving different students different tasks to complete within the same classroom, either because they are ahead of their peers or because they are struggling. The question that TALIS asked teachers was, "How often do you give different work to the students who have difficulties learning and/or to those who can advance faster?" The percentage refers to the number of teachers who said that they do this frequently or in almost all lessons.
I obtained the following graph:
From Liz Truss's comments, I expected some countries with high performance to have low levels of differentiation. However, I also expected the graph to have no overall shape with other high performing countries using this strategy a great deal. I didn't expect to see the rough negative trend that appears to be present.
We have to be careful here, this is merely a correlation. We cannot even say that it is a particularly strong one although it's not bad for the social sciences. I do not have the skill to be able to assign margins of error to the data and so we have to view it all with a large dose of healthy scepticism. Indeed, it is worth pointing out that England and France have wildly different responses to the survey question and yet possess similar PISA scores.
However, if this really does point to a negative association between this form of differentiation and achievement then what could possibly be the mechanism. Surely, providing students with work at a level suited to them has to be better than a one-size-fits-all approach. Indeed, I recently attended the Australian Council for Education Research (ACER) conference where the head of ACER, Geoff Masters, made the case for more personalisation of educational experiences.
One possible explanation for this data is to do with opportunity cost. What does a teacher not do to make room for the additional planning time required for producing differentiated work? Perhaps they don't spend time thinking of explanations or producing compelling tasks. What does the teacher not do when they are moving from group to group in a classroom, managing many disparate tasks? Perhaps they do not spend time at the front explaining concepts and conducting whole class discussions. And perhaps, with all of these different tasks taking place, the teachers need to spend more time on administering this activity and dealing with classroom management.
It's worth thinking about, perhaps.