Assessment

Progress 8 – Schools’ flexible friend?

Eight years after Progress 8 came into existence it is facing calls to be updated. Some say reforming the schools accountability measure can halt the decline in take up of the arts, while others want it to boost numbers taking vocational and technical subjects. Are its shoulders broad enough to take on such a load? In the first of two blogs on Progress 8, AQi looks at how it came into being, what problems it was designed to solve and whether it is flexible enough to be reformed.

Date11/01/24

AuthorAQi team

Well designed accountability measures can help students and schools grow

In 2013, the then Education Secretary Michael Gove announced he was reforming GCSEs to equip more young people with the knowledge and skills needed to succeed in their educational and work journey.

His plans for a successful transition included replacing the existing schools accountability measure.

Out went the 5ACEM scale which counted the number of pupils achieving grade C or better in five GCSEs, including maths and English.

In its place stood a combination of two new measures - Progress 8 and Attainment 8 - to assess the quality of secondary school teaching.

They came into force in 2016, marking the most important change to the schools accountability system since performance tables were first published in 1992.

But what was so wrong with 5ACEM that it needed to be completely ditched instead of simply adjusted?

Well, the main fault with 5ACEM was that because it measured the numbers getting grade C and above, schools had the perverse incentive of focusing on those pupils on the grade C/D borderline.

Pushing them over the boundary to get a grade C counted towards league table positions whereas helping someone move from an E to a D or a B to an A had no impact on that.

There were a couple of other perverse incentives linked to 5ACEM.

Firstly, it was said to encourage schools to choose exams that were seen as easy to pass rather than what were best for the students.

Secondly, because only five GCSEs (including maths and English) counted towards league table purposes, some schools narrowed their focus to just five subjects instead of the broader curriculum.

Another concern was that as it did not show how much students had improved against expectations, teachers’ hard work helping them produce better performances went unrecognised.

That meant good schools serving more disadvantaged communities were disproportionately hit by negative inspections.

At the same time, poorly performing schools which benefited from high-attaining intakes or were focused on getting lots of pupils over the C/D grade boundary fell under the inspectors’ radar.

The bottom line was that 5ACEM said more about the types of pupils taught in different schools than the effectiveness of education provided by those schools.

Progress 8 was designed to address those issues.

Before we get into that, here’s a quick explainer of how Progress 8 and Attainment 8 work.

Attainment 8 measures a student’s attainment across eight KS4 subjects - maths and English, three EBacc subjects and three other qualifications. The figure is the sum of the student’s results in those eight subjects.

Right, now the calculation part.

The Progress 8 score is the student’s Attainment 8 score number minus their Estimated Attainment 8 score (created at KS2), divided by 10.

A negative number signifies decline, a positive one shows progress.
A school’s Progress 8 is the mean average of all its students.
Failing to hit a threshold target on Progress 8 triggers an Ofsted inspection.

Since its introduction, educationalists have broadly seen Progress 8 as a fairer measure and having a positive effect as it incentivises teachers to help all students improve no matter where they are on the achievement spectrum.

Proof of this is shown in one study which found that once 5ACEM was removed, students outside the C/D borderline group seemed to make more relative progress compared to those at the borderline.

Clearly that is a positive for Progress 8 but it has not been all plain sailing.

It quickly emerged that a very small number of low achievers can distort a school’s Progress 8 results. This generally affects schools in the most disadvantaged areas.

It led to allegations of off-rolling by schools, where lower-performing pupils were removed from the register before sitting GCSEs so they did not impact on league table positions.

As a result, two sets of league tables are now published with one adjusted for the statistical impact of the small number of negative outliers.

The overall issue of off-rolling pupils is being looked at separately by the DfE.

Some also criticise it saying that because it only uses SATs to differentiate between pupils it is biased against schools serving more disadvantaged children.

The counter-argument is that adjusting for characteristics such as gender, ethnicity and free school meals would not necessarily improve measures of school effectiveness.

But, the main criticism of Progress 8 is that it incentivises Ebacc subjects at the expense of the non-Ebacc, especially the creative and vocational.

Although creative and vocational subjects are in the Progress 8 third grouping from which students pick their GCSEs, the way attainment is measured, it is weighted in favour of the traditional academic subjects of the Ebacc.

Not only that, quite a few vocational qualifications that could have gone in the third group have been defunded and don’t exist anymore. That means more traditional GCSEs are being taken instead.

All this has led to concerns that post-16 students are inadequately prepared for further and higher education and the workplace.

In our next blog we will be looking at whether Progress 8 can be adjusted to address these concerns and if so, how?

Assessment

Assessing oracy: Is Comparative Judgement the answer?

Oracy skills are vital to success in school and life. And yet, for many children, opportunities to develop them are missed. Educationalists are engaging in a growing debate about where oracy fits into the school system. Labour has put it at the heart of its plans to improve social mobility and an independent commission is looking at how it is taught in the classroom. This renewed focus on oracy means it is more important than ever that teachers have a way to reliably assess and understand their students’ attainment and progression. Amanda Moorghen of oracy education charity Voice 21 explains how Comparative Judgement can help with that and why it may be a game changer.

Education

TV subtitles as an aid to literacy: What does the research say?

Jack Black is probably best known in educational circles for playing a renegade substitute teacher in School of Rock. But the Hollywood star has made a more conventional foray into education by backing the use of TV subtitles to improve child literacy. Stephen Fry and the World Literacy Foundation also want parents to use their TV remotes to get children reading. So, could this simple click of a button be a solution to boost pupils’ reading skills? AQA’s resident expert on language teaching, Dr Katy Finch, casts her eye over the research to see whether it stacks up.

Data Analysis

What is left behind now education’s Data Wave has receded?

Is data the solution to all education’s issues? About a decade ago the prevailing wisdom said it was. Advocates of this Data Wave argued that harvesting internal statistics would help schools solve issues such as teacher accountability and attainment gaps. As with all waves, after crashing onto the beach they recede, leaving space for another to roll in. In this blog, teacher, author and data analyst Richard Selfridge looks at the legacy of the Data Wave to see what schools can take from it.

International Approaches

Finland & PISA – A fall from grace but still a high performer?

Finland was once recognised as one of the most successful educational systems in the world. At the turn of the millennium, it topped the PISA rankings in reading, maths and science. But by 2012, decline set in. The last set of results showed performances in maths, reading and science were at an all-time low. In this blog Dr Jonathan Doherty of Leeds Trinity University outlines some reasons that may account for the slide.

Briefing

PISA, TIMSS and PIRLS: What actually are they and what do they tell us?

According to the latest PISA results, England’s science scores are still on a downward trajectory that started a decade ago. Yet TIMSS, another respected study, has science performances rising. Which of them is right? Is one more valid than the other? In this blog AQi examines three International Large-Scale Assessments and finds that, although they may look the same from a distance, get up close and you’ll find they are very different beasts.

Adaptive Assessment

Adaptive Assessment: A missing ingredient in the resit recipe?

The number of students resitting their maths GCSE is growing, but the proportion getting a grade 4 or higher is falling. This situation is not only dispiriting for the young people striving to get the qualifications they need, but also for the teachers working hard to help them. How can outcomes for this cohort be improved? Bart Crisp, associate director at the Centre for the Use of Research and Evidence in Education, thinks adaptive assessment may be part of the solution.

SEND

Student success: Every milestone matters

Baroness Morgan is calling for students to be given ‘self-belief’ lessons as a way of developing their characters and preparing them for the future. She is not the first to notice that a student’s sense of their own ability and their level of success are part of a virtuous circle. But how can teachers get the snowball rolling for students with SEND or in alternative provision? In this blog, former headteacher, John Tomsett, pulls out a swimming certificate he earned more than half a century ago to use as an inspiration for others.

Assessment

Progress 8 – How much can it flex?

Progress 8 has come in for criticism in recent months for the way it combines with the Ebacc to skew schools’ focus towards academic subjects at the expense of creative and technical options. Both Labour and a Lords committee have called for reform to boost uptake of declining subjects such as D&T, dance and drama. In the second of two blogs looking at Progress 8, AQi examines whether the accountability measure could be realigned to address these concerns and if so, how could that be done?

Assessment