It is fair to say that any accountability measure in education brings about its own pressures, particularly on school leaders who are seeking good exam results.
There is always talk about where schools can make quick wins and boost their scores. Over the last few years the term ‘gaming’ has become popular, referring to people trying to manipulate the system. The term might be recent; the practice is not. But ‘gaming’, like ‘off-rolling’, is a practice that needs addressing.
What do we mean by ‘gaming’?
Gaming happens when the decisions made about the curriculum favour the league tables, rather than the individual needs of pupils.
One common argument is that leaders are under so much pressure to improve progress measures, that they feel forced into ‘gaming’. However, in my experience, most schools do the best for their individual pupils and do not game the system.
It is an inspector’s job to ask questions in the interests of pupils and parents and I have been asking myself serious ones about some very obvious differences in the elements of some schools’ Progress 8 buckets.
Why do we sometimes see high scores in the Open bucket and not similar results in the other buckets? Some of the differences are alarming. For example, it’s not unheard of to see a 1.6 point difference between the Open element and English and mathematics. Where we see this, we need to ask what this means and why it is occurring.
Progress 8
To explain, for the non-edugeeks among us, Progress 8 divides different subjects into elements, or buckets, all of which are weighted differently in terms of scores.
Pupils’ performance is measured in their GCSE year in three different buckets of subjects. The results of the buckets together create a score known as their P8 score.
Their mathematics and English results are double-weighted and are in bucket one. Their best three results in EBacc subjects are counted in bucket two. And bucket three, also known as the Open bucket, has three further qualifications in it. The qualifications may include GCSEs, as well as Department for Education (DfE) approved non-GCSE qualifications, such as art, music, and until recently, the European Computer Driving Licence (ECDL).
The sum of all these results is the attainment score. This score is compared with that of pupils with the same KS2 attainment to create the P8 score.
The ECDL factor
When pupils are entered for inappropriate qualifications, or the curriculum is reduced, it affects results. Where there’s exploitation, the DfE, which oversees the accountability system, will usually try and close loopholes. An example of this was the removal of the ECDL from the list of approved qualifications.
The ECDL no longer counts towards the final exam scores (feel free to take the qualification if you wish, but it won’t count towards Progress 8 scores). As a result, this year’s exam results should be more balanced towards the other subjects in the non-Open buckets.
This doesn’t mean the issue has been solved though. Inspectors will continue to scrutinise unusual entry patterns and look for evidence that shows that leaders have entered pupils for similar qualifications with significant subject overlap.
In the past, schools with very high entries of the ECDL qualification did well, on average, as outlined in a fascinating piece of research by dataeducator. Their research found that 209 schools entered more than 95% of their cohort for it and 2,240 schools used this qualification to some extent in 2017.
The average points score for ECDL was 52 points (equal to a grade A). Schools with high levels of entry, not coincidentally, tended to have very rosy Open P8 scores:
| Progress 8 overall | English element | Maths element | EBacc element | Open element |
| +0.23 | -0.09 | +0.01 | -0.06 | +0.88 |
Within this sample, there are schools that would have achieved a positive P8 score without this help, and very well done to them. But there are many that would not have.
Open group subjects: what is making the difference to grades?
We need to get behind the reasons why pupils in some schools are achieving higher marks in the Open element. On the face of it, either the quality of teaching in the non-Open elements is poor, or something remarkable is happening with regards to the Open bucket. Of course, it could be something else.
In inspection terms, this provokes some interesting questions. Is the teaching in the Open subjects, such as arts and vocational areas, strong? Or, in the schools concerned, is the teaching in other subjects very weak? (Given that the same children are performing far less well in other subjects.) Or is it something else? If so, what?
If pupils are making not just adequate, but amazing progress in the Open element, can they really be performing poorly in most of their other subjects? The answer could be ‘yes’, but there might be other factors involved. It seems to me imperative that we find out what is behind this.
I would want to know, if a school is doing so well at ensuring pupils gain great grades in the Open subjects, why leaders and teachers are not able to make the same difference to their learning in English and mathematics.
Inspection and the EBacc
Another area of accountability that inspectors look at is whether schools are working towards the EBacc target. We need to take into account the options available to pupils.
Inspectors will not expect school leaders to present separate plans about the EBacc, or to provide extra information outside of their normal curriculum planning. But they will ask leaders about their curriculum vision and the ambition for all their pupils. Inspectors will want to make sure that all pupils are receiving a breadth of knowledge to stand them in good stead for the future. You can read more about this in our recent school inspection update.
For me, as for most of us who choose to work in education, it is all about doing the best for pupils. In curriculum terms, this means making sure that they have a breadth of knowledge that will help them flourish and take a full role in society.
You can follow Emma on Twitter @EmmaIngHMI
You can keep up-to-date with Ofsted news by signing up for email alerts. You can also follow Ofsted on Twitter.
2 comments
Comment by Sajjad Ahmad posted on
Awesome Blog. Keep Updating people.
Comment by JP posted on
Open bucket qualifications include vocational qualifications that are graded based on criteria as opposed to GCSEs that are graded on cohort performance. Therefore teaching in vocational subjects can be seen to improve whilst in GCSEs it will appear to stay the same. Schools will then opt for the 'safe' option of a vocational qualification. Each year GCSE grades depend on how everyone else did.