I’m aware that some questions have recently been asked about our updated methodology for risk assessing good and outstanding schools and academies. Rather than attempt to answer every question individually via Twitter, I thought it would be better to respond to them all here.
First, I can understand why the words ‘machine learning algorithm’ gave some people a bit of a fright. But in reality, our risk assessment has changed very little since the previous methodology note was published.
Ofsted has risk assessed schools and academies for many years in order to help allocate inspection resource where it is most needed. It has never been used to pre-judge inspection grades.
The model has evolved over the years, as inspection frameworks and accountability measures have changed. The main change this time is a new statistical model, which we have found to work well within the data-analysis stage of our risk assessment process.
Like any modern organisation, we are keen to embrace the benefits of technology. But while it may sound ominous, ‘machine learning’ simply refers to a computer programme that helps us identify potential decline in a school, and that then re-jigs the underlying algorithm when inspection outcomes are known. It doesn’t mean we’re now using computers to make decisions without any human intervention, or indeed to judge schools. As before, Senior HMIs in the eight Ofsted regions will always thoroughly review the selection of schools for inspection and well-trained, experienced school inspectors will inspect on site.
So what do we use the risk assessment process for then? Well, as our handbook states, some good schools will automatically receive a full section 5 inspection instead of a section 8 short inspection. This occurs when our risk assessment process indicates that the quality of provision may have deteriorated significantly. Outstanding primary and secondary schools are of course exempt from routine inspection. However, if the risk assessment raises concerns about the performance of an exempt school, then it may also be inspected.
The new computer model uses progress and attainment data from the Department for Education, enhanced with school workforce census data and Parent View responses, to produce scores for each school, ranging from the lowest risk up to the highest risk. These scores are on a continuous scale, so there are no thresholds that automatically determine that a school should be inspected.
Of course, inspection outcomes will always be based on the evidence gathered on site. So to avoid them having undue influence, inspectors are not given the findings of the risk assessment.
The basic ideas behind the risk assessment process are outlined in our inspection handbook. We wrote the methodology note for those of you who like to know more detail about the statistical methodology involved. If it sounds a little like the rise of the machines, I can assure you that it’s really not like that at all.