Time to put PISA in its place?

December 5th 2016

Victoria Winckler argues that whatever the news we should take the PISA results – launched tomorrow – with a pinch of salt.


Once again there’s an outbreak of PISA fever – a rare disease that appears every three years. Journalists and opposition politicians are especially prone to catch it.  The symptoms are hysteria, loss of judgement and finger-pointing. The disease is caused by the release of the OECD’s findings from its most recent ‘Programme for International Student Assessment’ – and with the 2015 results due out tomorrow (6th December) the disease is running rife throughout Wales.

The PISA results excite attention because they provide an international comparator of performance at age 15.  Unlike comparisons of GCSE results, they are not only comparable across the UK but show how pupils are doing in everywhere from Finland to France and Brazil to Belgium. So they inevitably grab headlines in a way not dissimilar to rugby internationals.

Whether they tell us much more than ‘Wales beat Argentina’ – or not – is a moot point.

For a start, PISA is not quite as perfect an assessment as many make out. There are many educational experts who are deeply critical of technical aspects of PISA, including the lack of transparency of its statistical methods, difficulty of comparisons between very different societies and cultures as well as OECD’s partnership with certain agencies who stand to benefit from the findings.

It’s worth having a look at the questions OECD asks, not least as many look like tests of pupils’ ability to read and follow instructions rather than their wider skills – check out the example of a ‘problem solving exercise’ taken from the OECD website below.


Second, PISA values some aspects of learning more than others. The tests measure students’ ability in science, maths, reading, problem-solving and financial literacy. They say nothing about students’ creative abilities, their ability to express themselves, or indeed their happiness and well-being. One well-known commentator has accused OECD of ‘romanticising misery’ by holding up Shanghai-China as the beacon to which we should all aspire.

It’s worth comparing what PISA measures with the objectives of the Welsh curriculum which are:

  • ambitious, capable learners
  • healthy, confident individuals
  • enterprising, creative contributors
  • ethical, informed citizens.

Ask yourself if the PISA tests are measuring these qualities.

Third, PISA offers no explanations or solutions. It provides a stick with which to beat governments – and the Welsh Government is by no means alone it getting a whacking – but it doesn’t explain why one country does worse than other nor offer any remedies.

Frankly, the deeper you delve into PISA the less and less useful and meaningful the tests appear. Stop and think for a moment – why is the idea that the whole of a country’s education system can be summed up in a single score credible?

I am not arguing that we should ignore PISA. Clearly the test results tell us something about how Wales’ 15 year olds are doing, just as any snap shot test can be a useful indicator. Wales’ education system is far from perfect, with too many young people leaving school without the magic clutch of 5 GCSEs at A* to C.

But Cabinet Secretary for Education, Kirsty Williams should hold firm – a single, dubious test should not drive Wales’ education policy.

Victoria Winckler is Director of the Bevan Foundation


Users who have LIKED this post:

  • avatar

Leave a Reply

In Print

Subscribers' Magazine

Exchange – Issue 7

‘Resilient communities’ is a phrase that’s been hard to avoid in recent months and is central to the discussions about the future of Communities First.  Great though it sounds, it is much less clear what resilience means or how it Read more »

Exchange – Issue 6

It’s been almost five months since the UK voted to leave the EU. As the initial shock of the referendum result dies down, the latest edition of Exchange looks at the impact that Brexit could have on Wales. Our contributors Read more »

More from the Subscribers' Magazine »

Other Publications

More from Other Publications »


Keep up to Date

Sign up to our monthly e-news to keep up to date with our ideas, events and resources.

Support Us

Subscribe Today!

Help Wales to be fair, prosperous and sustainable - and get some great benefits too!

Join / Donate »


June 14th 2017

What next after the election?

Whatever the result of the election on June 8th, big changes are ahead over the next five years.  Come and join us for a great lunch-time discussion! We are pleased to invite you to discuss the results of the general Read more »

July 5th 2017

Making City Deals Inclusive

Making City Deals Inclusive Wednesday 5th July 2017 Media Briefing Room, Senedd, Cardiff, CF99 1NA Kindly sponsored by Jeremy Miles AM With City Deals now approved for Cardiff and Swansea – and similar deals on the agenda for the rest of Read more »