Best Practices for Open Book Testing

Reading Time: 4 minutes

Are you ready to refresh the assessments in your synchronous or online asynchronous courses? As you weigh the option of an open-book test, this teaching tip will provide best practices and considerations for creating the most effective open-book assessment possible.

Is open-book better than closed-book to measure learning?

Perhaps reassuringly, research evidence of a measurable difference in student performance between open-book and closed-book exams is so far inconclusive (Durning, et al., 2016; Stowell, 2015), although Minder, et al., found open-book formative assessment to be effective in preparing medical students for upcoming practical courses (2018). There are also inconclusive reports about whether an open-book test reduces test-taking anxiety (Durning, et al., 2016; Johanns, et al., 2017). 

What are the advantages of an open-book assessment?

Open-book exams can reduce the incentive to cheat because students are allowed to check information sources without penalty (Johanns, et al., 2017).  Additionally, well-formed exam questions encourage students to apply course content meaningfully, encouraging the use of higher order thinking skills (HOTS) (Stowell, 2015).

What does Open-Book / Open-Resource mean?

An open-book assessment has a few basic characteristics.

• Students leverage information resources available to them as they demonstrate their ability to synthesize, evaluate, and analyze content. ​
• An open-book test is versatile.  It can take the form of an in-person test, an online quiz, or a take-home assessment.
• An open-book assessment can consist of one cohesive assessment submission or serve as one part of a modularized test.

What goes into an open-book test?

Similar to any type of assessment, an effective open-book test requires the following:

• Clear instructions and protocols​ so students know exactly what is expected
• An evaluation tool (marking scheme or rubric)​ explaining how work will be assessed
• An eConestoga Quiz to take OR an assignment dropbox where students can submit the assessment

Considerations in Creating an open-book test

• Identify the course learning outcomes you are assessing as part of the test.
• Focus on higher order thinking skills such as application, evaluation, and synthesis (Bengtsson, 2019). Consider sample question stems from “The Spectrum of Effective Questioning” to help guide your test construction.
• Ensure that questions require thoughtful application.
• For longer questions such as case studies and scenarios, consider providing students with choice.
• Will students be permitted to use AI to assist them in completing the open book test?  
  • If so, state specifically how students can use AI as an assistive technology. For example, “Use of generative AI platforms, including (but not limited to) chatbots, coding tools, and image/art generating tools, is not permitted in the completion of this assessment, with the exception of: a. Brainstorming and idea generation; b. Thesis refinement; c. Proofreading; d. (Add/Remove components as desired).” 
  • If you wish to forbid AI use in the open book test, communicate that using language such as, “Use of any form of generative AI is strictly forbidden throughout all stages of this assessment, including brainstorming, idea generation, outline formation, etc. Generative AI may include (but is not limited to) chatbots, coding tools and image/art generating tools, or other technologies that produce work for you.”
  • To stay one step ahead of AI-generated responses, consider questions that call for summarizing, comparing, and contrasting content accurately across multiple sources or drawing connections between texts and visuals or videos. Ask students to ground their responses in course-specific discussions, contexts, tasks, and sources. Add opportunities for personalized responses. 
• Try testing out your questions on a colleague to ensure that they are clear.
• Ensure that students have been exposed to the kinds of questions they will find on the test beforehand through in-class work and/or test preparation tasks (Durning, et al., 2016).
• Set a reasonable time limit which includes time to think about each question, but not time to look up every question to formulate the answer.  Students should still need to study to be successful.
• Decide whether students need to cite sources and clearly explain these expectations.
• Decide on the modality of the test. You can create an eConestoga quiz or you can have students write the assessment in a word document which is then uploaded to an assignment dropbox. This allows you to activate turn-it-in to double-check for academic integrity.
• Clarify what open-book means for your test.  Given that there is no reliable way with Respondus Lockdown Browser, Respondus Monitor, or Faronics Insight to successfully allow sole access to an online resource while taking a test, it is best to concede that an online open-book test is essentially an open-internet test as well. However, it is still important to identify what is allowed and what is not. Talk about academic integrity with your students in relation to an open-book test.
• Share exam instructions with students well in advance of the test. Include suggestions about the best way to prepare for the test. For example, you can recommend that students prepare a short summary of key terms/formulas/concepts in preparation for the test. This will encourage studying for the test and help students not fall into the trap of not preparing because they think that for an open-book assessment they don’t need to do so. 
• Consider doing activities with students in class to prepare study notes for the test throughout the semester. Find ways to help students to prepare resources to support them as they take their test. This can include modeling and supporting the preparation of glossaries, summaries, visual aids, etc. This will not only help set students up to succeed but will also encourage them to put in study time getting their notes ready—so learning will take place whether or not they are tested on each topic.

Further Supports

If you’d like to talk further about creating effective assessments, contact teachingandlearning@conestogac.on.ca

References

Bengtsson, L. (2019). Take-home exams in higher education: A systematic review. Education Sciences, 9(4), 267. https://doi.org/10.3390/educsci9040267

Durning, S. J., Dong, T., Ratcliffe, T., Schuwirth, L., Artino, A. R., Boulet, J. R., & Eva, K. (2016). Comparing open-book and closed-book examinations. Academic Medicine, 91(4), 583–599. https://doi.org/10.1097/acm.0000000000000977

Johanns, B., Dinkens, A., & Moore, J. (2017). A systematic review comparing open-book and closed-book examinations: Evaluating effects on development of critical thinking skills. Nurse Education in Practice, 27, 89–94. https://doi.org/10.1016/j.nepr.2017.08.018

Minder, S., Weibel, D., Wissmath, B., and Schmitz,  F. (2018). Do students achieve the desired learning goals using open-book formative assessments? Int J Med Educ., 9, 293-301. doi: 10.5116/ijme.5bc6.fead. PMID: 30457974; PMCID: PMC6387773.

Stowell, J. R. (2015). Online open-book testing in face-to-face classes. Scholarship of Teaching and Learning in Psychology, 1(1), 7–13. https://doi.org/10.1037/stl0000014