Wales' most innovative and influential think tank

Connect: Twitter Facebook LinkedInTel: 01685 350938

Time to put PISA in its place?

December 5th 2016

Victoria Winckler argues that whatever the news we should take the PISA results – launched tomorrow – with a pinch of salt.


Once again there’s an outbreak of PISA fever – a rare disease that appears every three years. Journalists and opposition politicians are especially prone to catch it.  The symptoms are hysteria, loss of judgement and finger-pointing. The disease is caused by the release of the OECD’s findings from its most recent ‘Programme for International Student Assessment’ – and with the 2015 results due out tomorrow (6th December) the disease is running rife throughout Wales.

The PISA results excite attention because they provide an international comparator of performance at age 15.  Unlike comparisons of GCSE results, they are not only comparable across the UK but show how pupils are doing in everywhere from Finland to France and Brazil to Belgium. So they inevitably grab headlines in a way not dissimilar to rugby internationals.

Whether they tell us much more than ‘Wales beat Argentina’ – or not – is a moot point.

For a start, PISA is not quite as perfect an assessment as many make out. There are many educational experts who are deeply critical of technical aspects of PISA, including the lack of transparency of its statistical methods, difficulty of comparisons between very different societies and cultures as well as OECD’s partnership with certain agencies who stand to benefit from the findings.

It’s worth having a look at the questions OECD asks, not least as many look like tests of pupils’ ability to read and follow instructions rather than their wider skills – check out the example of a ‘problem solving exercise’ taken from the OECD website below.


Second, PISA values some aspects of learning more than others. The tests measure students’ ability in science, maths, reading, problem-solving and financial literacy. They say nothing about students’ creative abilities, their ability to express themselves, or indeed their happiness and well-being. One well-known commentator has accused OECD of ‘romanticising misery’ by holding up Shanghai-China as the beacon to which we should all aspire.

It’s worth comparing what PISA measures with the objectives of the Welsh curriculum which are:

  • ambitious, capable learners
  • healthy, confident individuals
  • enterprising, creative contributors
  • ethical, informed citizens.

Ask yourself if the PISA tests are measuring these qualities.

Third, PISA offers no explanations or solutions. It provides a stick with which to beat governments – and the Welsh Government is by no means alone it getting a whacking – but it doesn’t explain why one country does worse than other nor offer any remedies.

Frankly, the deeper you delve into PISA the less and less useful and meaningful the tests appear. Stop and think for a moment – why is the idea that the whole of a country’s education system can be summed up in a single score credible?

I am not arguing that we should ignore PISA. Clearly the test results tell us something about how Wales’ 15 year olds are doing, just as any snap shot test can be a useful indicator. Wales’ education system is far from perfect, with too many young people leaving school without the magic clutch of 5 GCSEs at A* to C.

But Cabinet Secretary for Education, Kirsty Williams should hold firm – a single, dubious test should not drive Wales’ education policy.

Victoria Winckler is Director of the Bevan Foundation


Users who have LIKED this post:

  • avatar

Leave a Reply

In Print

Supporter Magazine

Exchange Issue 10, Spring 2018

“Refreshing, timely and balanced – Exchange has its finger on the pulse” The latest edition of our exclusive supporters’ magazine brings together a range of contributors to explore some of the most current and important issues in Welsh public life. Read more »

Exchange – Issue 9

Well-being in Wales The latest edition of our exclusive supporters’ magazine looks at well-being in Wales from a range of different perspectives. Our excellent range of contributors looks at health and social care in Wales while considering different aspects of Read more »

More from the Supporter Magazine »

Other Publications

Response to the Green Paper on Local Government

The Bevan Foundation has submitted its response to the Green Paper on local government reform. The Bevan Foundation’s response to the Welsh Government’s Green Paper ‘Strengthening Local Government, Delivering for People‘ argues that the Welsh Government needs to address the Read more »

More from Other Publications »


Stay informed

Keep up to date and gain new insights and inspiration with Spark, our FREE e-news

Sign Up For The Spark »

Support Us

Support lasting solutions today!

We need YOUR help to create equality, justice and prosperity for everyone in Wales

Support Us »


June 22nd 2018

Community Film Screening: I, Daniel Blake

The Bevan Foundation and Dove Workshop present I, Daniel Blake as part of Adult Learners’ Week 2018 “… a powerful, beautiful drama about dignity in the face of humiliation.” Friday, 22nd June 2018 at Dove Workshop 6.00 Refreshments    6.30 Film Read more »

June 26th 2018

Community Film Screening: I, Daniel Blake

The Bevan Foundation and Merthyr Valley Homes present I, Daniel Blake as part of Adult Learners’ Week 2018 “… a powerful, beautiful drama about dignity in the face of humiliation.” Tuesday, 26th June 2018 12.30 Refreshments          1.00 Film starts    3.00 Read more »