Assessment

Open-book exams: what does the evidence say?

Amid debate around the merits of open-book exams, AQi examines some of the evidence.

Date25/03/22

AuthorAndrew Quentin, Researcher, AQA

When you think of exams, you probably imagine rows of students in a hall, heads down, scribbling away in silence. The only things on their desks are the exam paper, a blue or black pen, and – at most – a calculator or other specialist equipment needed for the exam. There are certainly no notes or textbooks in sight.

Well, there is another way. Open-book exams allow students to consult set texts, notes or the internet in their exam to construct their answers. They can draw on all the information they need to inform their answers, rather than relying on knowledge recall.

There have been calls from some quarters for open-book exams to become the norm. Open-book exams, it is argued, could encourage more creative and critical thinking. That they are a valid and fair way to assess a student’s skills and understanding. This is a topic that has provoked some heated discussions recently.

Let’s take a look at what some of the available evidence says about open-book exams.

What is the impact on student performance?

There appears to be limited research on this topic, particularly when it comes to the specific context of UK high-stakes assessments. Also, given that many participants in previous studies had little or no experience of open-book exams, any reported effects on performance might be due to this lack of experience rather than the test format.

Where researchers directly compared the open-book and closed-book exam formats, they found no significant differences in student performance. Evidence that students do better in closed-book exams[1] has been attributed to them spending less time preparing for open-book exams[2], because they think they’ll be able to find answers in the sources they have at hand.[3]

Moreover, when it comes to expectations that open-book exams may promote more deep learning, these haven’t necessarily been supported in studies.[4]

While there may be limited evidence of performance differences, there are some advantages and disadvantages to using each of these formats.

What are the pros and cons of open-book exams?

An open-book exam can help assess skills such as analysis and evaluation, looking beyond knowledge recall.[5] It can also replicate the sort of tasks that students might be expected to perform in the workplace, such as finding, synthesising and interpreting information from multiple sources.[6] [7]

In some cases, students may prefer open-book exams to closed-book exams as they allow them to make more creative use of course content and have a greater sense of ownership over their personal study.[8] Students also appear to associate open-book exams with less anxiety[9], (even if, in practice, 45% of students reported equivalent levels of stress when taking exams in either format).[10]

On the flip side, there is a risk that open-book exams might fail to assess students’ ability to understand and recall information.

Some studies report that students consider open-book exam questions to be more difficult and would like more training on how to complete this type of exam.[11] Perhaps not surprisingly, where students have less experience with open-book exams, their grades are initially lower until they develop specific skills for this exam format[12].

Another consideration with open-book exams is that students can spend a lot of time finding information instead of formulating their answers.[13] This means they either need longer to complete the exam – potentially creating fatigue issues – or the exam needs to contain fewer questions. However, reducing the number of questions may impact the reliability of the students’ results.[14]

A tailored approach

On the whole, it appears a sensible approach would be to ensure that the exam format suits the purpose of the assessment. Broadly speaking, closed-book exams are useful when assessing students’ knowledge (and can also be used to assess higher-order skills) while open-book exams are useful where the main goal is to assess students’ ability to analyse and evaluate information.

One clear conclusion is that any transition from closed-book exams to open-book exams would need to be carefully managed to ensure that students understand what is expected of them.

References

[1] Durning, S. J., Dong, T., Ratcliffe, T., Schuwirth, L., Artino, A. R., Boulet, J. R., & Eva, K. (2016). Comparing open-book and closed-book examinations: a systematic review. Academic Medicine, 91, 583–599

[2] Block, R. M. (2012). A discussion of the effect of open-book and closed-book exams on student achievement in an introductory statistics course. Problems, Resources, and Issues in Mathematics Undergraduate Studies, 22(3), 228–238.

[3] Dale, V. H., Wieland, B., Pirkelbauer, B., & Nevel, A. (2009). Value and benefits of open-book examinations as assessment for deep learning in a post-graduate animal health course. Journal of Veterinary Medical Education, 36, 403–410.

[4] Heijne-Penninga, M., Kuks, J. B. M., Hofman, W. H. A., & Cohen-Schotanus, J. (2008). Influence of open- and closed-book tests on medical students' learning approaches. Medical Education, 42(10), 967–974.

[5] Green, S. G., Ferrante, C. J. & Heppard, K. A. (2016). Using Open-Book Exams to enhance Student Learning, Performance, and Motivation. The Journal of Effective Teaching, 16, 19–35.

[6] Deneen, C. (2020). Assessment considerations in moving from closed-book to open-book exams (Melbourne CSHE Teaching and Learning short guide series). University of Melbourne.

[7] Williams, J. B., & Wong, A. (2009). The efficacy of final examinations: A comparative study of closed‐book, invigilated exams and open‐book, open‐web exams. British Journal of Educational Technology, 40, 227–236.

[8] Brightwell, R., Daniel, J. H., & Stewart, A. (2004). Evaluation: Is an open book examination easier? Bioscience Education, 3(1), 1–10.

[9] Durning, S. J., Dong, T., Ratcliffe, T., Schuwirth, L., Artino, A. R., Boulet, J. R., & Eva, K. (2016). Comparing open-book and closed-book examinations: a systematic review. Academic Medicine, 91, 583–599

[10] Baillie C., & Toohey, S. (1997). The “power test”: Its impact on student learning in a materials science course for engineering. Assessment & Evaluation in Higher Education, 22, 33–48.

[11] Dale, V. H., Wieland, B., Pirkelbauer, B., & Nevel, A. (2009). Value and benefits of open-book examinations as assessment for deep learning in a post-graduate animal health course. Journal of Veterinary Medical Education, 36, 403–410.

[12] Durning, S. J., Dong, T., Ratcliffe, T., Schuwirth, L., Artino, A. R., Boulet, J. R., & Eva, K. (2016). Comparing open-book and closed-book examinations: a systematic review. Academic Medicine, 91, 583–599

[13] Theophilides, C., & Dionysiou, O. (1996). The major functions of the open-book test at the university level: A factor analytic study. Studies in Educational Evaluation, 22, 157–170.

[14] Durning, S. J., Dong, T., Ratcliffe, T., Schuwirth, L., Artino, A. R., Boulet, J. R., & Eva, K. (2016). Comparing open-book and closed-book examinations: a systematic review. Academic Medicine, 91, 583–599

Assessment

Assessing oracy: Is Comparative Judgement the answer?

Oracy skills are vital to success in school and life. And yet, for many children, opportunities to develop them are missed. Educationalists are engaging in a growing debate about where oracy fits into the school system. Labour has put it at the heart of its plans to improve social mobility and an independent commission is looking at how it is taught in the classroom. This renewed focus on oracy means it is more important than ever that teachers have a way to reliably assess and understand their students’ attainment and progression. Amanda Moorghen of oracy education charity Voice 21 explains how Comparative Judgement can help with that and why it may be a game changer.

Education

TV subtitles as an aid to literacy: What does the research say?

Jack Black is probably best known in educational circles for playing a renegade substitute teacher in School of Rock. But the Hollywood star has made a more conventional foray into education by backing the use of TV subtitles to improve child literacy. Stephen Fry and the World Literacy Foundation also want parents to use their TV remotes to get children reading. So, could this simple click of a button be a solution to boost pupils’ reading skills? AQA’s resident expert on language teaching, Dr Katy Finch, casts her eye over the research to see whether it stacks up.

Data Analysis

What is left behind now education’s Data Wave has receded?

Is data the solution to all education’s issues? About a decade ago the prevailing wisdom said it was. Advocates of this Data Wave argued that harvesting internal statistics would help schools solve issues such as teacher accountability and attainment gaps. As with all waves, after crashing onto the beach they recede, leaving space for another to roll in. In this blog, teacher, author and data analyst Richard Selfridge looks at the legacy of the Data Wave to see what schools can take from it.

International Approaches

Finland & PISA – A fall from grace but still a high performer?

Finland was once recognised as one of the most successful educational systems in the world. At the turn of the millennium, it topped the PISA rankings in reading, maths and science. But by 2012, decline set in. The last set of results showed performances in maths, reading and science were at an all-time low. In this blog Dr Jonathan Doherty of Leeds Trinity University outlines some reasons that may account for the slide.

Briefing

PISA, TIMSS and PIRLS: What actually are they and what do they tell us?

According to the latest PISA results, England’s science scores are still on a downward trajectory that started a decade ago. Yet TIMSS, another respected study, has science performances rising. Which of them is right? Is one more valid than the other? In this blog AQi examines three International Large-Scale Assessments and finds that, although they may look the same from a distance, get up close and you’ll find they are very different beasts.

Adaptive Assessment

Adaptive Assessment: A missing ingredient in the resit recipe?

The number of students resitting their maths GCSE is growing, but the proportion getting a grade 4 or higher is falling. This situation is not only dispiriting for the young people striving to get the qualifications they need, but also for the teachers working hard to help them. How can outcomes for this cohort be improved? Bart Crisp, associate director at the Centre for the Use of Research and Evidence in Education, thinks adaptive assessment may be part of the solution.

SEND

Student success: Every milestone matters

Baroness Morgan is calling for students to be given ‘self-belief’ lessons as a way of developing their characters and preparing them for the future. She is not the first to notice that a student’s sense of their own ability and their level of success are part of a virtuous circle. But how can teachers get the snowball rolling for students with SEND or in alternative provision? In this blog, former headteacher, John Tomsett, pulls out a swimming certificate he earned more than half a century ago to use as an inspiration for others.

Assessment

Progress 8 – How much can it flex?

Progress 8 has come in for criticism in recent months for the way it combines with the Ebacc to skew schools’ focus towards academic subjects at the expense of creative and technical options. Both Labour and a Lords committee have called for reform to boost uptake of declining subjects such as D&T, dance and drama. In the second of two blogs looking at Progress 8, AQi examines whether the accountability measure could be realigned to address these concerns and if so, how could that be done?

Assessment

Progress 8 – Schools’ flexible friend?

Eight years after Progress 8 came into existence it is facing calls to be updated. Some say reforming the schools accountability measure can halt the decline in take up of the arts, while others want it to boost numbers taking vocational and technical subjects. Are its shoulders broad enough to take on such a load? In the first of two blogs on Progress 8, AQi looks at how it came into being, what problems it was designed to solve and whether it is flexible enough to be reformed.

Assessment

2023: A year in review

As we wrap up a year full of exciting developments in education, it is the perfect time to take stock of what has gone on. In this blog, editor of AQi, Anil Dawar, reflects on some of his highlights for 2023.

Download a PDF version.

Download a copy of this content to your device as a PDF file. We generate PDF versions for the convenience of offline reading, but we recommend sharing this link if you'd like to send it to someone else.

Open-book exams: what does the evidence say?

What is the impact on student performance?

What are the pros and cons of open-book exams?

A tailored approach

References

Assessing oracy: Is Comparative Judgement the answer?

TV subtitles as an aid to literacy: What does the research say?

What is left behind now education’s Data Wave has receded?

Finland & PISA – A fall from grace but still a high performer?

PISA, TIMSS and PIRLS: What actually are they and what do they tell us?

Adaptive Assessment: A missing ingredient in the resit recipe?

Student success: Every milestone matters

Progress 8 – How much can it flex?

Progress 8 – Schools’ flexible friend?

2023: A year in review

Join the conversation on Twitter

Download a PDF version.

Sign up to our newsletter