Assessment

Strictly Come Dancing and assessment or: How I learned to stop worrying and love Craig Revel Horwood

You may see Strictly Come Dancing is a bit of TV glitz and glamour that enlivens Saturday nights for millions. Or, if you were so inclined, you may view the perennial favourite as a weekly two-part dance exam based on expert judgement and a populist vote. In this blog, originally published in October 2015, assessment expert Kate Kelly explains what she loves about the show and why the ‘Mr Nasty’ judge Craig Revel Horwood is crucial to its fairness.

Date21/09/23

AuthorKate Kelly

Craig Revel Horwood is crucial to fairness on Strictly Come Dancing
Photo: BBC

It's definitely autumn: the leaves are turning, the mornings are darker and the Pumpkin Spice Latte has returned to Starbucks. For me, this can only mean one thing: a new series of Strictly Come Dancing!

This might seem like an odd topic for a research blog post, but all competitions have some form of assessment at their heart.

As much as I love the glitter, the dancing and the dodgy puns, I'm also fascinated by the parallels between Strictly and the work we do in AQA’s research unit.

One such parallel comes from the scoring system.

What is the Strictly Come Dancing scoring system?

For those not in the know, this is how it works.
Each of the four judges gives every dance a mark out of 10, leading to a total score out of 40.

Once all the contestants have danced, the public can vote for their favourite.
The highest ranked contestant in the public vote is given a score equal to the number of couples competing and so on in descending order.

Each contestant is then assigned two scores - one based on the judge's scores, one based on their popularity in the public vote.
These are added together to form an overall score the two contestants with the lowest combined score are entered into the dance-off.

So, whether or not a contestant goes into the dance-off is half determined by their scores from the judges, and half by the public vote.

How does Strictly Come Dancing's scoring system relate to assessment?

In the same way, a GCSE or A-level candidate's final grade is usually based on two or more assessments, each of which contributes a certain percentage of the overall marks.

We could, therefore, think of each week of Strictly as a dance exam comprising two components, each contributing 50% to the final outcome.

This contribution is known as an intended weighting.

Why is discrimination in Strictly Come Dancing scoring vital?

But, whether an intended weighting is achieved or not depends on the extent to which each component discriminates between the contestants.

Let's illustrate this with an extreme example.

Imagine that every contestant on Strictly Come Dancing was given the same score by the judges.

In this scenario, all contestants have the same place in the judges' rank order. The judges have not discriminated at all between the contestants.

As can be seen in the table below, when you combine these rankings with the rank from the public vote, each contestant's place in the final rank order is determined entirely by the public vote. (The voting data is not published, so the public vote rank order here is entirely fictional.)

What happens if there is no discrimination in scoring contestants?

By failing to discriminate between the contestants, the judges' scores don't contribute at all to the final outcomes and therefore have an actual weighting of 0%.

By the same token, if every couple had the same number of public votes, their place in the final rank order would be entirely influenced by the judges' scores.

Obviously, such an extreme scenario is unlikely to occur, but it neatly demonstrates how the variation in each set of rank orders can affect how much they contribute to the overall outcome.

It's for this reason (among others) that good discrimination is one of the features we look for in our assessments, and we routinely monitor the achieved weightings of our examinations.

Why Craig Revel Horwood should win the assessors' vote

It's also for this reason that Craig Revel Horwood is my favourite Strictly Come Dancing judge.

Known for being Strictly's Mr Nasty, Craig has a reputation for being the harshest judge, regularly dishing out twos and threes where the others give fours and fives.

But while Len Goodman, Dame Darcey Bussell and Bruno Tonioli might be nicer, by rarely giving marks below four they aren't distinguishing between dancers as clearly.

Craig's willingness to give out lower scores increases the range of total scores, discriminating better between contestants.

This in turn helps to ensure that the judges' scores make a contribution to the overall outcomes.

So, if you've ever complained that Strictly is nothing more than a popularity contest, don't blame Craig.

This blog was originally published in October 2015 before the death of Len Goodman in April 2023

Education Policy

Through the looking glass: How polling the public can help policymakers learn about themselves

Public attitude data is key to effective policymaking. Proper polling can reveal what people think about existing policies and what they want for the future. But, if looked at from a different angle, it can also help policymakers question themselves and their assumptions about the public. In this blog, AQA’s Policy and Evidence Manager Adam Steedman-Thake, reveals the lessons he learned about himself while reading a recent public attitude survey.

Assessment

Assessing oracy: Is Comparative Judgement the answer?

Oracy skills are vital to success in school and life. And yet, for many children, opportunities to develop them are missed. Educationalists are engaging in a growing debate about where oracy fits into the school system. Labour has put it at the heart of its plans to improve social mobility and an independent commission is looking at how it is taught in the classroom. This renewed focus on oracy means it is more important than ever that teachers have a way to reliably assess and understand their students’ attainment and progression. Amanda Moorghen of oracy education charity Voice 21 explains how Comparative Judgement can help with that and why it may be a game changer.

Education

TV subtitles as an aid to literacy: What does the research say?

Jack Black is probably best known in educational circles for playing a renegade substitute teacher in School of Rock. But the Hollywood star has made a more conventional foray into education by backing the use of TV subtitles to improve child literacy. Stephen Fry and the World Literacy Foundation also want parents to use their TV remotes to get children reading. So, could this simple click of a button be a solution to boost pupils’ reading skills? AQA’s resident expert on language teaching, Dr Katy Finch, casts her eye over the research to see whether it stacks up.

Data Analysis

What is left behind now education’s Data Wave has receded?

Is data the solution to all education’s issues? About a decade ago the prevailing wisdom said it was. Advocates of this Data Wave argued that harvesting internal statistics would help schools solve issues such as teacher accountability and attainment gaps. As with all waves, after crashing onto the beach they recede, leaving space for another to roll in. In this blog, teacher, author and data analyst Richard Selfridge looks at the legacy of the Data Wave to see what schools can take from it.

International Approaches

Finland & PISA – A fall from grace but still a high performer?

Finland was once recognised as one of the most successful educational systems in the world. At the turn of the millennium, it topped the PISA rankings in reading, maths and science. But by 2012, decline set in. The last set of results showed performances in maths, reading and science were at an all-time low. In this blog Dr Jonathan Doherty of Leeds Trinity University outlines some reasons that may account for the slide.

Briefing

PISA, TIMSS and PIRLS: What actually are they and what do they tell us?

According to the latest PISA results, England’s science scores are still on a downward trajectory that started a decade ago. Yet TIMSS, another respected study, has science performances rising. Which of them is right? Is one more valid than the other? In this blog AQi examines three International Large-Scale Assessments and finds that, although they may look the same from a distance, get up close and you’ll find they are very different beasts.

Adaptive Assessment

Adaptive Assessment: A missing ingredient in the resit recipe?

The number of students resitting their maths GCSE is growing, but the proportion getting a grade 4 or higher is falling. This situation is not only dispiriting for the young people striving to get the qualifications they need, but also for the teachers working hard to help them. How can outcomes for this cohort be improved? Bart Crisp, associate director at the Centre for the Use of Research and Evidence in Education, thinks adaptive assessment may be part of the solution.

SEND

Student success: Every milestone matters

Baroness Morgan is calling for students to be given ‘self-belief’ lessons as a way of developing their characters and preparing them for the future. She is not the first to notice that a student’s sense of their own ability and their level of success are part of a virtuous circle. But how can teachers get the snowball rolling for students with SEND or in alternative provision? In this blog, former headteacher, John Tomsett, pulls out a swimming certificate he earned more than half a century ago to use as an inspiration for others.

Assessment

Progress 8 – How much can it flex?

Progress 8 has come in for criticism in recent months for the way it combines with the Ebacc to skew schools’ focus towards academic subjects at the expense of creative and technical options. Both Labour and a Lords committee have called for reform to boost uptake of declining subjects such as D&T, dance and drama. In the second of two blogs looking at Progress 8, AQi examines whether the accountability measure could be realigned to address these concerns and if so, how could that be done?

Assessment

Progress 8 – Schools’ flexible friend?

Eight years after Progress 8 came into existence it is facing calls to be updated. Some say reforming the schools accountability measure can halt the decline in take up of the arts, while others want it to boost numbers taking vocational and technical subjects. Are its shoulders broad enough to take on such a load? In the first of two blogs on Progress 8, AQi looks at how it came into being, what problems it was designed to solve and whether it is flexible enough to be reformed.

Download a PDF version.

Download a copy of this content to your device as a PDF file. We generate PDF versions for the convenience of offline reading, but we recommend sharing this link if you'd like to send it to someone else.

Strictly Come Dancing and assessment or: How I learned to stop worrying and love Craig Revel Horwood

What is the Strictly Come Dancing scoring system?

How does Strictly Come Dancing's scoring system relate to assessment?

Why is discrimination in Strictly Come Dancing scoring vital?

What happens if there is no discrimination in scoring contestants?

Why Craig Revel Horwood should win the assessors' vote

Through the looking glass: How polling the public can help policymakers learn about themselves

Assessing oracy: Is Comparative Judgement the answer?

TV subtitles as an aid to literacy: What does the research say?

What is left behind now education’s Data Wave has receded?

Finland & PISA – A fall from grace but still a high performer?

PISA, TIMSS and PIRLS: What actually are they and what do they tell us?

Adaptive Assessment: A missing ingredient in the resit recipe?

Student success: Every milestone matters

Progress 8 – How much can it flex?

Progress 8 – Schools’ flexible friend?

Join the conversation on Twitter

Download a PDF version.

Sign up to our newsletter