Focused mark schemes make question writing clearer and exams fairer
Photo: Scott Graham/Unsplash

From the outside, people probably think marking an exam is easy. Subject experts write questions and similarly qualified exam markers then award points according to how well students answer them. The harder the question the more marks they get, right?

The truth is far more complicated than that. AQA has to ensure hundreds of examiners mark thousands of student answers to the same standard. Here AQA’s Deputy Head of Assessment Design, Suzanne Oates, reveals the complexities she and her colleagues face and how her department’s work improves student outcomes.

Mark scheme first?

In training, assessment writers are encouraged to write mark schemes before they write the question.
‘How can you write a mark scheme before knowing what the question is?’ I hear you cry.

Well, it's very easy to think you've written a question which is clear and focused on what it is testing, but then inadvertently create a mark scheme crediting a broader range of knowledge than has actually been targeted.

If a question targeting one aspect of student knowledge is accompanied by a mark scheme crediting a wider understanding, then examiners will be confused.

They could query how to mark this response, leading to delays and uncertainty.
They may mark by the letter of the scheme and not award marks for a clearly correct answer (to the question that has been asked) or determine that the response given is good enough and award a mark.

All of these are unfair outcomes.

Why the mark scheme should come first

Writing the mark scheme first, clearly outlining how we want students to respond, helps define the question they need to ask.

It's not as simple as 'yes' or 'no'

Most exams are more complicated than just having right or wrong answers.
English, Drama, Psychology, even Computer Science assessments can be complex and prompt a vast array of answers for the same question.
Examiners have to know what they are meant to be looking for in a response.

Has the student identified the right information? Do they show a detailed understanding of the subject matter?

The myriad ways students could do this needs to be included  in the mark scheme.
It is necessarily a complex guide that must be clearly constructed to ensure it can be applied fairly.
We must carefully create a ‘levels of response’ mark scheme that identifies the qualities expected in a response.

Do we want to see analysis of language used or application of theories? How do we differentiate between good and excellent analysis?

There also needs to be some indicative content. We know the kinds of things students are likely to refer to in responses so we make it clear to examiners what kinds of content are worthy of credit.
And there has to be room for students to make points we might not have even thought of!

Not just a tick box exercise

What’s important with mark schemes is that we reward the quality of the responses rather than just crediting use of concepts or key phrases students have learned.

Correct application of these concepts and phrases is what must be rewarded to ensure fair and accurate outcomes.

Getting this right has huge implications for the fairness of our exams.

How reviewing mark schemes helps improve exam fairness

We always review our mark schemes looking for improvements.

One things we actively seek to eliminate are redundant marks - marks students can’t get because of the way schemes operate.

A good example is a question such as ‘State the colours of the rainbow in the correct order.’ [8 marks] accompanied by the mark scheme - 1 mark per correct colour in its correct place and 1 mark for completely correct answer

In this example the student cannot score 7 marks.

If any of the colours are missing or in the wrong order, then they would be able to achieve a maximum of 6 marks. If they have all of the correct colours and they are in the correct order then they would be awarded 8 marks.

So, one mark is, essentially, redundant. It represents a wasted opportunity to differentiate between students and their abilities.

A far better approach in this case would be for it to be worth seven marks instead of eight - with the seventh mark being awarded for getting the correct order - and applying a cap of six to any student who makes an error.

So next time you hear about exam marking, don’t assume it’s just putting a tick next to an answer, the thinking around it goes a lot deeper than that.

Read more on this topic:

AQA | AQA Assessment Research and Innovation | Research library | Inside Assessment 2019
Making the grades – How does an exam turn into a qualification? | AQi powered by AQA
Journey of a script – What happens after students put their pens down? | AQi powered by AQA