Time to put PISA in its place?

People Children standing in a row
ViewsDecember 5th, 2016

Victoria Winckler argues that whatever the news we should take the PISA results – launched tomorrow – with a pinch of salt.

Once again there’s an outbreak of PISA fever – a rare disease that appears every three years. Journalists and opposition politicians are especially prone to catch it.  The symptoms are hysteria, loss of judgement and finger-pointing. The disease is caused by the release of the OECD’s findings from its most recent ‘Programme for International Student Assessment’ – and with the 2015 results due out tomorrow (6th December) the disease is running rife throughout Wales.

The PISA results excite attention because they provide an international comparator of performance at age 15.  Unlike comparisons of GCSE results, they are not only comparable across the UK but show how pupils are doing in everywhere from Finland to France and Brazil to Belgium. So they inevitably grab headlines in a way not dissimilar to rugby internationals.

Whether they tell us much more than ‘Wales beat Argentina’ – or not – is a moot point.

For a start, PISA is not quite as perfect an assessment as many make out. There are many educational experts who are deeply critical of technical aspects of PISA, including the lack of transparency of its statistical methods, difficulty of comparisons between very different societies and cultures as well as OECD’s partnership with certain agencies who stand to benefit from the findings.

It’s worth having a look at the questions OECD asks, not least as many look like tests of pupils’ ability to read and follow instructions rather than their wider skills – check out the example of a ‘problem solving exercise’ taken from the OECD website below.

pisa-question

Second, PISA values some aspects of learning more than others. The tests measure students’ ability in science, maths, reading, problem-solving and financial literacy. They say nothing about students’ creative abilities, their ability to express themselves, or indeed their happiness and well-being. One well-known commentator has accused OECD of ‘romanticising misery’ by holding up Shanghai-China as the beacon to which we should all aspire.

It’s worth comparing what PISA measures with the objectives of the Welsh curriculum which are:

  • ambitious, capable learners
  • healthy, confident individuals
  • enterprising, creative contributors
  • ethical, informed citizens.

Ask yourself if the PISA tests are measuring these qualities.

Third, PISA offers no explanations or solutions. It provides a stick with which to beat governments – and the Welsh Government is by no means alone it getting a whacking – but it doesn’t explain why one country does worse than other nor offer any remedies.

Frankly, the deeper you delve into PISA the less and less useful and meaningful the tests appear. Stop and think for a moment – why is the idea that the whole of a country’s education system can be summed up in a single score credible?

I am not arguing that we should ignore PISA. Clearly the test results tell us something about how Wales’ 15 year olds are doing, just as any snap shot test can be a useful indicator. Wales’ education system is far from perfect, with too many young people leaving school without the magic clutch of 5 GCSEs at A* to C.

But Cabinet Secretary for Education, Kirsty Williams should hold firm – a single, dubious test should not drive Wales’ education policy.

Victoria Winckler is Director of the Bevan Foundation

Leave a Reply

Search

Search and filter the archive using any of the following fields:

  • Choose Type:

  • Choose Focus:

  • Choose Tag:

Close