Assessments with Purpose: Leveraging Test Blueprints to Organize Question Libraries and Optimize Question Pool Selection

Reading Time: 5 minutes

As educators, we all know how challenging it can be to design or select assessment questions for our courses. Whether it’s for quizzes, mid-terms, or final exams, deciding what content to include, how to format questions, and ensuring alignment with learning outcomes—while also maintaining equity—can feel overwhelming. If this resonates with you, you’re not alone. Research shows that many faculty members face similar struggles when trying to craft assessments that align effectively with course learning outcomes and content (Rubeba & William, 2023; Zubairi, 2015).

In such situations, it’s common to fall back on simpler methods, such as combing through course materials to create questions based on the easiest content to formulate. Or, if you’re using eConestoga, you might find yourself randomly selecting questions from question libraries. While these approaches may seem like quick fixes, they often result in significant drawbacks. Assessments created this way are frequently misaligned with learning outcomes, include redundant or trivial questions, lack balanced coverage of course content (e.g., focusing too heavily on one section while neglecting others), and fail to assess a range of cognitive skills effectively. These issues undermine the reliability and validity of assessments, meaning they fail to accurately reflect what students have learned. As a result, students may feel frustrated, unprepared, or even unfairly evaluated.

One useful approach that you can take to address this problem is by creating a Table of Specifications (TOS) or Test Blueprint. A TOS is a tool that helps ensure assessments align with learning outcomes by organizing and balancing questions across different content areas and cognitive levels. This strategy provides a clear framework for developing fair and comprehensive assessments that accurately measure students’ knowledge and skills. Using a TOS ensures that your assessments (a) cover a well-rounded mix of content across your course, (b) match the cognitive levels required by your learning outcomes, (c) avoid redundancy and trivial questions, and (d) fairly represent the course material.

In this post, we’ll explore how you can create and use your own test specification table to design, select, or even build question pools that align with your course learning outcomes.

Table of Specification/ Test Blueprint
A Table of Specification or Test blueprint, or table of specifications, is a structured guide used to design tests. It outlines the topics and skills the test will cover and matches them with the learning outcomes. This ensures the test is well-organized, covers all important areas, and aligns with what was taught in class and the intended learning goals (DiDonato-Barnes et al, 2014; Grondlund,2006; Reynolds et al., 2006). The following sections discusses the steps involved in creating a test blueprint.

Steps in Creating Test Blueprint

Step 1: Identify Course Learning Outcomes
a. List the course learning outcomes: Start by identifying all the course learning outcomes or objectives for your course. These outcomes should describe what students are expected to know or be able to do after completing the course. Example: ” Describe teaching frameworks that support active learning,” ” Explain how to use the BOPPPS lesson planning model to create active learning environments,” or ” Discuss how to positively incorporate AI as a learning and assessment resource.”

b. Categorize learning outcomes by domains of learning: Use Bloom’s Taxonomy (or a similar framework approved by the College) to categorize each outcome into cognitive domains. Decide on the range of cognitive levels to assess for each learning outcome. For instance, knowledge-based outcomes may require recall or identification. Skills-based outcomes may require application or analysis. Higher-order outcomes may require evaluation or creation. If some learning outcomes are more critical to your course than others, make a note of their relative importance. This prioritization will influence the weight given to different outcomes in your test

Step 2: Analyze the Course Content
a. Break down the content: Divide your course content into specific topics or units that directly align with the learning outcomes. Example: For an ‘Active Learning’ Course, the content might include “Active learning for diverse students,” and “Assessment opportunities for diverse students”.

b. Identify key concepts: Highlight the main ideas, principles, and skills students should master in each content area. This should be based on the course or unit learning outcomes. Each topic corresponds to at least one learning outcome. If a topic does not align with any outcome, it should not be included in the assessment.

Step 3: Define the Test Format
a. Choose question types: Determine the types of questions to include, such as:
o Multiple-choice questions (MCQs).
o Short answer questions.
o Essay questions.
o Problem-solving tasks.
o Case studies or scenarios.
Note: The question format should align with the learning outcomes.

b. Determine the number of questions: Decide the total number of questions on the test based on the allotted time and difficulty level.
c. Allocate time per question type: Estimate the time needed for students to complete each type of question to ensure the test can be completed within the allotted time.

Step 4: Assign Weight to Each Assessment Task
Proportionate Weighting: Allocate weights to each assessment task. You can use the weights allocated on your course outlines to guide you.

Step 5: Create the Test Specification Table
Structure the Table: Use a grid format with rows and columns.

Sample of Test Blueprint for an Active Learning Course

Unit/Topic	Learning Objective	Cognitive Level (Bloom’s Taxonomy)	Weight (%)	Number of Items	Item Type
1.0 Active learning for diverse students			50%	10
1.1 Teaching frameworks	Describe teaching frameworks that support active learning.	Understand	10%	2	Multiple-choice
1.2 Intercultural sensitivity strategies	Discuss intercultural sensitivity strategies to create inclusive learning opportunities.	Apply	10%	2	Short answer
1.3 Accessible learning practices	Identify practices that foster accessible learning opportunities for all learners.	Understand	10%	2	Scenario-based question
1.4 Classroom management techniques	Recognize classroom management techniques and institutional supports to maintain a positive environment.	Understand	10%	2	Multiple-choice
1.5 BOPPPS model for lesson planning	Explain how to use the BOPPPS lesson planning model to create active learning environments.	Apply	10%	2	Short answer
2.0 Assessment opportunities for diverse students			50%	10
2.1 Key assessment concepts	Describe key assessment concepts to support diverse learning needs.	Understand	10%	2	Multiple-choice
2.2 Assessment administration strategies	Outline strategies that support student accommodations and academic integrity at Conestoga.	Analyze	10%	2	Short answer
2.3 Rubrics and tools for feedback	Identify the value of rubrics and other tools in grading and feedback.	Understand	10%	2	Scenario-based question
2.4 Artificial intelligence in assessment	Discuss how to positively incorporate AI as a learning and assessment resource.	Evaluate	20%	4

Using the Test Blueprint to Develop or Select Test Items from Question Pools
Once you have created your test blueprint, you can use it to design or select questions from the Question Library in your course. The Question Library is where you build and store quiz questions. Once your questions are in the library, you can import them into one or more quizzes in your course. To stay organized, it’s essential to structure your question library from the beginning by creating Sections (like folders) to group questions. Your test blueprint will help you organize questions based on content coverage, difficulty level, and alignment with course learning outcomes. For example, you might categorize questions by format (e.g., short answers, Multiple choice, matching etc.), topic, or difficulty level. This organization is especially helpful if you plan to use Question Pools, where students receive a randomized selection of questions

The following short video by my colleague, Nathan Gardi from the Online Learning Centre, shows how to use the Table of Specifications to effectively organize your Question Library and ensure balanced representation.

Conclusion
Using a Test Blueprint is not just about structuring assessments—it’s about ensuring that your evaluations truly reflect what you want your students to learn. As faculty, we strive to create fair, meaningful, and well-balanced assessments that align with our course outcomes, and the blueprint serves as a practical tool to make that process more intentional and effective.
By taking the time to map out your assessments, you not only enhance the reliability and validity of your evaluations but also provide students with a clearer, more transparent learning experience. Thoughtfully designed assessments foster deeper engagement, reduce frustration, and give students a real opportunity to demonstrate their understanding. In the end, using a Test Blueprint helps you teach better, and your students learn better—and that’s a win for everyone.