A lot of work goes into putting grade boundaries in the right place
Photo: Erin Larson/Unsplash
Awarding students a grade is simple right? Exam marks are converted into levels of achievement and certificates issued.
But what if the system of determining those levels of achievement has been thrown up in the air by three years of disrupted assessment caused by lockdown?
Grade boundary creation before lockdown
Normally, GCSE grade boundaries are set using statistics from a previous year showing its GCSE outcomes against the expectations from same cohort’s KS2 results.
This helps to predict what should be expected from the current cohort against their KS2 scores.
For A-level assessments, grade boundaries are set exactly the same way but using GCSE scores as a measure of prior attainment instead.
The resulting statistical information, what we call ‘predictions’, shows what percentage of students should be getting each grade.
Mark Distributions are then used to work out where each boundary should fall.
After the predictions have been applied to the marks, an awarding committee of examiners looks at the completed and marked papers to check the scripts actually show performance at that standard.
For example, if the prediction suggests minimum mark for a grade A should be 32, the committee might look at scripts from 30-34.
This may lead to fine tuning if it’s decided the paper was very difficult or easy, or the cohort was more or less able than the reference year.
Coping with three years of disruption
So that’s a normal year, but how do you create fair grade boundaries when you have nothing recent to which they can be compared?
The lack of Summer exams in 2020 and 2021 meant awards were decided by Centre Assessed Grades and Teacher Assessed Grades, which led to higher outcomes compared to pre-pandemic standards.
Ofqual’s plan for retuning to ‘normal’ pre-lockdown standards was to use 2022’s exam series as a half step.
Grade Boundaries in 2022
Last year we used a brand-new performance standard – the midpoint between pre-pandemic 2019’s results and the Teacher Assessed Grades of 2021.
The system was justified with the argument that using 2019 performance standards alone would be unfairly high for students affected by three years of disrupted learning.
At the same time, relying solely on the higher 2020/2021 outcomes would not help the aim of moving closer to pre-pandemic standards.
As expected, the midpoint policy worked, with the 2022 outcomes higher than pre-pandemic 2019 but lower than the elevated outcomes of 2021.
Returning to pre-pandemic standards
In 2023, we are reverting to the ‘normal’ way of generating predictions and using only one reference year, 2019.
The aim is to maintain an existing performance standard - the pre-pandemic standard.
If exams were equally demanding from one year to the next, grade boundaries could remain the same.
But even subject experts are unable to precisely predict how difficult or easy an assessment will be, making it impossible to ensure they will be equally demanding.
That’s why new grade boundaries are set every time a subject is examined.
An assessment has to be taken before its difficulty is accurately known and comparable grade boundaries set.
It ensures the same level of performance is rewarded the same way every year and is why the proportion of students getting each grade every year is different.
What to expect from Grade Boundary setting this year
2023’s boundary predictions will be more heavily statistical than usual. But the role of human judgement is still very important.
A combination of examiners’ expertise and statistical evidence is the best way of setting standards.
Examiners are better at judging student based on overall performance than on performance samples.
Equally, statistics cannot take into account all elements of context – such as the impact of last year’s Advance Information.
We need to see what would happen if we applied the grade boundaries suggested by predictions and whether or not that would be an accurate assessment of student performance. Only a human can do this.
It is important we get it right as what happens this year is vital for ensuring the post-pandemic awarding system is just as stable as it was going into lockdown.
Read More on this subject:
Making the grades – How does an exam turn into a qualification? | AQi powered by AQA
Journey of a script – What happens after students put their pens down? | AQi powered by AQA
Making up an exam: How does a question paper get made? | AQi powered by AQA