Over recent days, there has been considerable interest in perceived bias in inspection judgements for schools that have high proportions of White British pupils who are eligible for free school meals (FSM).
This blog post from Sean and me expands the range of evidence in this debate, but recognises too that, given the complex interaction of schools’ circumstances and make-up, distilling the debate into a few simple charts is not straightforward. That said, we do seek to shine a light on several wider factors that contribute to Ofsted’s overall effectiveness judgement, particularly the provision of a good standard of education.
In this post, I will deal with the data issues – that’s my job! But I do need to stress that inspection is about much more than just data analysis. To look backwards for a moment, the analysis published prior to this blog was prepared in response to a very specific request from a member of the public. This request focused on the outcomes of schools that have high proportions of White British pupils who are eligible for FSM.
The broad assertion in some of the reporting of that data was that school judgements are negatively biased against schools with relatively high levels of deprived (as measured by those who are eligible for FSM) White British pupils. While Ofsted strongly refutes any accusation of systematic bias in our judgements, we are not surprised by findings that identify differences between overall effectiveness judgements in schools or areas with differing levels of progress and attainment.
So what does the data say?
First, let’s look at the original analysis. There were some misunderstandings about what the data showed – we addressed those in a separate piece of work sent to the original blogger - so we’ve replicated the original chart with clearer labelling below.
Most recent overall effectiveness grades of secondary schools, by the proportion of pupils who are White British and eligible for free school meals, 30 April 2018
In some of the coverage, the original analysis was interpreted as only including schools with a large proportion of White British pupils eligible for FSM. In fact, it showed all 3,158 secondary schools with the relevant data available.
It is worth saying too, that the previous analysis (in looking specifically at White British versus non-White British) provides only a limited reflection of UK society. This could deflect us away from the very real issues of under-performance in other groups of pupils, for example, Black Caribbean boys. While, ideally, we would like to look at more detailed differences across schools for all the major ethnic population groups, the smaller population sizes and the need for more complex pupil level analysis mean that cannot be addressed here.
So how does this link to the progress and attainment of White British pupils?
We see for example that nationally White British pupils eligible for FSM make worse progress in their time at secondary school than almost any other group. As illustrated in the table published by the Department for Education[1] below, their average Progress 8 score is minus 0.72.
Furthermore, only 35% of White British pupils eligible for FSM score a grade 4 or above in their English and maths GCSEs, compared with 64% of all pupils. So while accepting that inspections are about much more than data, it would be perverse to overlook such poor outcomes where we see them at inspection.
What does the progress data tell us about any potential bias in our
inspection outcomes?
Some have said that Ofsted just inspects 'by numbers' and that by using a non-contextualised Progress 8 score to do so, schools with higher proportions of poor White British pupils are at a disadvantage because the measure is biased against them. If we look at the Progress 8 scores for schools with varying proportions of White British pupils eligible for FSM, the data does not support this view. The following chart shows that for schools with higher proportions of White British pupils eligible for FSM, there are lower median Progress 8 scores for the same overall effectiveness judgements. For instance, schools with a low percentage of this group of pupils and that have been judged to be good have a median Progress 8 score of +0.2. Meanwhile for good schools with a high percentage of this group, there is a median Progress 8 score of -0.1. If there were bias based on progress expectations, we would expect the data to show similar levels of progress scores for each overall effectiveness judgement, regardless of the proportion of White British pupils eligible for FSM.
Median Progress 8 scores for secondary schools, by the proportion of pupils who are White British and eligible for free school meals, and the most recent overall effectiveness grade of the school, 30 April 2018 [2].
While some differences are relatively small there is a clear pattern. Rather than suggesting a bias against deprived schools, if anything, this data shows that inspectors are demonstrating, through their judgements, an awareness that Progress 8 isn’t a perfect measure of progress and that it doesn't paint the whole picture of educational quality in a secondary school.
If you have thoughts about what additional analysis would be helpful and constructive in this important area, please let us know.
Sean Harford HMI, National Director, Education
As Jason says, there has been a lot of interest in the idea that inspections are biased against schools with certain demographics. I know from my experience of teaching and from my 15 years as one of Her Majesty’s Inspectors working with schools in the most challenging circumstances that, for a number of reasons, working in schools in deprived areas is a tough gig. The teachers who do so have my utmost admiration and respect. Things have moved on considerably in schools across the country in the last decade and a half since I started inspecting. While in the early 2000s chair-chucking in schools in special measures was not an unusual occurrence, that blatant kind of outburst is now much less prevalent. That said, as we identified in our report ‘Below the radar’[3] in 2014, low-level disruption continues to be an issue in too many schools – and inspection data shows that this is disproportionately the case in schools in deprived areas. The drip, drip, drip of low-level disruptive behaviour saps teachers’ well-being and makes for an ineffective learning environment.
But we also know that there is a whole range of compounding issues and challenges for schools in deprived areas. Teacher recruitment and retention, not easy in many parts of the country, is especially difficult in deprived areas. Only this week, two separate reports[4] show that not only are schools in deprived areas more likely to employ less experienced teachers, but also those with lower levels of subject qualification.
Similarly, the recruitment of leaders and managers is more difficult in these areas. And this includes recruiting effective governors, which is particularly difficult when the communities are disaffected, monocultural and lack capacity.
Additionally, access to good resources in the community to support the quality of education continues to be more limited in deprived areas; this is particularly acute in poor coastal areas. How much easier is it to travel by London tube to some of the best galleries and museums in the world, compared with bussing pupils for hours to the capital at great cost in time and money?
Believe me, I am fully aware of the challenges faced by secondary schools in predominantly low socio-economic areas, where traditional industries often no longer exist. Considering the issues I have raised here, is it any wonder that there are around 260 more schools in the most deprived areas judged to be less than good compared with the least deprived areas?
But in a system where we want the best for all children, whatever their backgrounds, we cannot accept a lower quality of education, often caused by the factors I have noted, just because the schools are in deprived areas. The key point is that the overall effectiveness judgement, driven by the quality of education, is not an effort grade. If it were, I have no doubt that there would be many fewer schools judged to be less than good because every teacher and school I know works hard.
Let’s be clear, the majority of secondary schools with above average proportions of poor White British pupils are providing at least a good education for their pupils according to our judgements. We just need even more pupils to be getting that good deal so that they have the best chance to make progress, achieve and develop as young people. Let's identify where the quality of education needs to be better to ensure the system can provide the right support to make it better.
Further sources of information
[1] Revised GCSE and equivalent results in England: 2016 to 2017’, Department for Education, 2018
[2] Based on Ofsted analysis of pupil level data from the Department for Education, but related publications are available. ‘Revised GCSE and equivalent results in England: 2016 to 2017’, Department for Education, 2018. ‘Schools, Pupils and their characteristics: January 2017’, Department for Education, 2017
[3] ‘Low-level disruption in classrooms: below the radar’, Ofsted, 2014
[4] 'The more disadvantaged the pupil, the less experienced their maths teacher', Schoolsweek 2018 and 'Effective teacher policies', OECD 2018
8 comments
Comment by R N Sharma posted on
Good morning
Good news
still having plenty that they are well off and self respected of having food of their own in schools.
Please Enhance proper management , resources , open up industry , businesses by redeveloping and proper utilizing properties of religious places assets , Government assets as they have been created by people money.
Comment by Colin Richards posted on
Not really convincing. As a matter of justice all "effectiveness judgments" should be related to context. There is no set of objective Platonic standards that can be applied fairly to all schools . What's effective in one context isn't necessarily effective in another. Ofsted needs to clarify its thinking in this area as a precursor to changing its policy and practice. The new 2019 inspection framework offers that possibility.
Comment by Ofsted external relations posted on
Hi, As HMCI made clear in her speech at Wellington last week https://www.gov.uk/government/speeches/amanda-spielmans-speech-at-the-wellington-festival-of-education Ofsted’s job is to provide an objective assessment of the quality of education, because all children and young people deserve a strong education built around an effective curriculum, regardless of the setting and context in which they find themselves. Judging the quality of education is not a contextual exercise. Nevertheless, as HMCI also made clear in the same speech, Ofsted recognises the greater challenges that teachers and leaders face in schools in challenging areas. Our inspections act as a call for support in those areas where standards are consistently lower. It is important that this is reflected in other parts of Ofsted’s thinking and reporting.
Comment by G Lowe posted on
I am concerned that 'ethnicity' is automatically linked to an article about deprivation and underachievement. Why is that? Does this indicate a 'racial lense' through which OFSTED operates?
Comment by C Wheston posted on
re table 1 above, if I use this data to calculate the average scores of the non-white British pupils, then the c. 25% of secondary pupils who aren't white British are performing much better than their white British counterparts, both if receiving FSM, and also if not. Is this the correct conclusion, and if so, has Ofsted done any work to establish why this is?
Comment by Ofsted external relations posted on
Hi, The table you refer to is taken from data published by the Department for Education: http://www.gov.uk/government/statistics/revised-gcse-and-equivalent-results-in-england-2016-to-2017. If you look at the file ‘Characteristics national tables: SFR01/2018’, Table CH2a, you can see progress 8 scores for each major ethnic group, by free school meals eligibility.
Comment by Kay Kempers posted on
What happens if you balance out the percentage of FSM into evenly-sized portions instead of using an equal number of schools in each slot? One category accounts for a 7% spread and another is 43%. I’m curious how it would look.
Comment by Eshal Fatima posted on
Great Post. I really like your post.