Assessment

Grade Boundaries: Getting back to ‘normal’ after the pandemic

2023’s exam series will be the first ‘normal’ set for three years because of effects of the lockdown. Confidence in certification standards is based on having last year’s grades help determine this year’s. But what if you do not have that recent frame of reference. Here, in the first of a series of blogs from those involved in sitting, setting and assessing the exams, AQA’s Standards and Awarding Manager Elizabeth Pope reveals how her unit’s work got back to ‘normal’.

Date20/04/23

AuthorElizabeth Pope

A lot of work goes into putting grade boundaries in the right place
Photo: Erin Larson/Unsplash

Awarding students a grade is simple right? Exam marks are converted into levels of achievement and certificates issued.

But what if the system of determining those levels of achievement has been thrown up in the air by three years of disrupted assessment caused by lockdown?

Grade boundary creation before lockdown

Normally, GCSE grade boundaries are set using statistics from a previous year showing its GCSE outcomes against the expectations from same cohort’s KS2 results.

This helps to predict what should be expected from the current cohort against their KS2 scores.

For A-level assessments, grade boundaries are set exactly the same way but using GCSE scores as a measure of prior attainment instead.

The resulting statistical information, what we call ‘predictions’, shows what percentage of students should be getting each grade.

Mark Distributions are then used to work out where each boundary should fall.

After the predictions have been applied to the marks, an awarding committee of examiners looks at the completed and marked papers to check the scripts actually show performance at that standard.

For example, if the prediction suggests minimum mark for a grade A should be 32, the committee might look at scripts from 30-34.

This may lead to fine tuning if it’s decided the paper was very difficult or easy, or the cohort was more or less able than the reference year.

Coping with three years of disruption

So that’s a normal year, but how do you create fair grade boundaries when you have nothing recent to which they can be compared?

The lack of Summer exams in 2020 and 2021 meant awards were decided by Centre Assessed Grades and Teacher Assessed Grades, which led to higher outcomes compared to pre-pandemic standards.

Ofqual’s plan for retuning to ‘normal’ pre-lockdown standards was to use 2022’s exam series as a half step.

Grade Boundaries in 2022

Last year we used a brand-new performance standard – the midpoint between pre-pandemic 2019’s results and the Teacher Assessed Grades of 2021.

The system was justified with the argument that using 2019 performance standards alone would be unfairly high for students affected by three years of disrupted learning.

At the same time, relying solely on the higher 2020/2021 outcomes would not help the aim of moving closer to pre-pandemic standards.

As expected, the midpoint policy worked, with the 2022 outcomes higher than pre-pandemic 2019 but lower than the elevated outcomes of 2021.

Returning to pre-pandemic standards

In 2023, we are reverting to the ‘normal’ way of generating predictions and using only one reference year, 2019.

The aim is to maintain an existing performance standard - the pre-pandemic standard.

If exams were equally demanding from one year to the next, grade boundaries could remain the same.

But even subject experts are unable to precisely predict how difficult or easy an assessment will be, making it impossible to ensure they will be equally demanding.

That’s why new grade boundaries are set every time a subject is examined.

An assessment has to be taken before its difficulty is accurately known and comparable grade boundaries set.

It ensures the same level of performance is rewarded the same way every year and is why the proportion of students getting each grade every year is different.

What to expect from Grade Boundary setting this year

2023’s boundary predictions will be more heavily statistical than usual. But the role of human judgement is still very important.

A combination of examiners’ expertise and statistical evidence is the best way of setting standards.

Examiners are better at judging student based on overall performance than on performance samples.

Equally, statistics cannot take into account all elements of context – such as the impact of last year’s Advance Information.

We need to see what would happen if we applied the grade boundaries suggested by predictions and whether or not that would be an accurate assessment of student performance. Only a human can do this.

It is important we get it right as what happens this year is vital for ensuring the post-pandemic awarding system is just as stable as it was going into lockdown.

Education Policy

Through the looking glass: How polling the public can help policymakers learn about themselves

Public attitude data is key to effective policymaking. Proper polling can reveal what people think about existing policies and what they want for the future. But, if looked at from a different angle, it can also help policymakers question themselves and their assumptions about the public. In this blog, AQA’s Policy and Evidence Manager Adam Steedman-Thake, reveals the lessons he learned about himself while reading a recent public attitude survey.

Assessment

Assessing oracy: Is Comparative Judgement the answer?

Oracy skills are vital to success in school and life. And yet, for many children, opportunities to develop them are missed. Educationalists are engaging in a growing debate about where oracy fits into the school system. Labour has put it at the heart of its plans to improve social mobility and an independent commission is looking at how it is taught in the classroom. This renewed focus on oracy means it is more important than ever that teachers have a way to reliably assess and understand their students’ attainment and progression. Amanda Moorghen of oracy education charity Voice 21 explains how Comparative Judgement can help with that and why it may be a game changer.

Education

TV subtitles as an aid to literacy: What does the research say?

Jack Black is probably best known in educational circles for playing a renegade substitute teacher in School of Rock. But the Hollywood star has made a more conventional foray into education by backing the use of TV subtitles to improve child literacy. Stephen Fry and the World Literacy Foundation also want parents to use their TV remotes to get children reading. So, could this simple click of a button be a solution to boost pupils’ reading skills? AQA’s resident expert on language teaching, Dr Katy Finch, casts her eye over the research to see whether it stacks up.

Data Analysis

What is left behind now education’s Data Wave has receded?

Is data the solution to all education’s issues? About a decade ago the prevailing wisdom said it was. Advocates of this Data Wave argued that harvesting internal statistics would help schools solve issues such as teacher accountability and attainment gaps. As with all waves, after crashing onto the beach they recede, leaving space for another to roll in. In this blog, teacher, author and data analyst Richard Selfridge looks at the legacy of the Data Wave to see what schools can take from it.

International Approaches

Finland & PISA – A fall from grace but still a high performer?

Finland was once recognised as one of the most successful educational systems in the world. At the turn of the millennium, it topped the PISA rankings in reading, maths and science. But by 2012, decline set in. The last set of results showed performances in maths, reading and science were at an all-time low. In this blog Dr Jonathan Doherty of Leeds Trinity University outlines some reasons that may account for the slide.

Briefing

PISA, TIMSS and PIRLS: What actually are they and what do they tell us?

According to the latest PISA results, England’s science scores are still on a downward trajectory that started a decade ago. Yet TIMSS, another respected study, has science performances rising. Which of them is right? Is one more valid than the other? In this blog AQi examines three International Large-Scale Assessments and finds that, although they may look the same from a distance, get up close and you’ll find they are very different beasts.

Adaptive Assessment

Adaptive Assessment: A missing ingredient in the resit recipe?

The number of students resitting their maths GCSE is growing, but the proportion getting a grade 4 or higher is falling. This situation is not only dispiriting for the young people striving to get the qualifications they need, but also for the teachers working hard to help them. How can outcomes for this cohort be improved? Bart Crisp, associate director at the Centre for the Use of Research and Evidence in Education, thinks adaptive assessment may be part of the solution.

SEND

Student success: Every milestone matters

Baroness Morgan is calling for students to be given ‘self-belief’ lessons as a way of developing their characters and preparing them for the future. She is not the first to notice that a student’s sense of their own ability and their level of success are part of a virtuous circle. But how can teachers get the snowball rolling for students with SEND or in alternative provision? In this blog, former headteacher, John Tomsett, pulls out a swimming certificate he earned more than half a century ago to use as an inspiration for others.

Assessment

Progress 8 – How much can it flex?

Progress 8 has come in for criticism in recent months for the way it combines with the Ebacc to skew schools’ focus towards academic subjects at the expense of creative and technical options. Both Labour and a Lords committee have called for reform to boost uptake of declining subjects such as D&T, dance and drama. In the second of two blogs looking at Progress 8, AQi examines whether the accountability measure could be realigned to address these concerns and if so, how could that be done?

Assessment

Progress 8 – Schools’ flexible friend?

Eight years after Progress 8 came into existence it is facing calls to be updated. Some say reforming the schools accountability measure can halt the decline in take up of the arts, while others want it to boost numbers taking vocational and technical subjects. Are its shoulders broad enough to take on such a load? In the first of two blogs on Progress 8, AQi looks at how it came into being, what problems it was designed to solve and whether it is flexible enough to be reformed.

Download a PDF version.

Download a copy of this content to your device as a PDF file. We generate PDF versions for the convenience of offline reading, but we recommend sharing this link if you'd like to send it to someone else.

Grade Boundaries: Getting back to ‘normal’ after the pandemic

Grade boundary creation before lockdown

Coping with three years of disruption

Grade Boundaries in 2022

Returning to pre-pandemic standards

What to expect from Grade Boundary setting this year

Through the looking glass: How polling the public can help policymakers learn about themselves

Assessing oracy: Is Comparative Judgement the answer?

TV subtitles as an aid to literacy: What does the research say?

What is left behind now education’s Data Wave has receded?

Finland & PISA – A fall from grace but still a high performer?

PISA, TIMSS and PIRLS: What actually are they and what do they tell us?

Adaptive Assessment: A missing ingredient in the resit recipe?

Student success: Every milestone matters

Progress 8 – How much can it flex?

Progress 8 – Schools’ flexible friend?

Join the conversation on Twitter

Download a PDF version.

Sign up to our newsletter