Faculty Innovation 2022 – a Faculty Member’s Story 2

Reading Time: 6 minutes

The theme of the 2022 Faculty Innovation Award was “inspiring students to engage meaningfully in assessment.”  We met with Amanda Tkaczyk, the award winner from the School of Applied Computer Science and Information Technology, to talk about Scaffolded Carousel Assessments. Given that she was the only one teaching the course that semester, Amanda was in a unique position to try an alternative format for the assessments. This is what she told us.

What was the opportunity that inspired you to innovate?

While preparing to teach PROG 8630 – Introduction to Data Visualization and Reporting, I learned that this was the second cohort of students in the newly launched post-graduate certificate program and I met with the previous instructor to get feedback on how the course might be improved for future offerings.  This motivated me to do a mini-literature review and develop ideas to continue to improve the course in alignment with the existing course outline. This innovation was centered around 3 design objective themes to engage students meaningfully in assessment: increasing online student engagement, reducing academic integrity threat, and improving group work.

First, there were challenges in online student engagement  as we recovered from Covid.  Some of the challenges that the research literature have shown include fatigue during classes and lack of participation. However, there is evidence of increased engagement when online breakout groups have a specific task to achieve. Given this, I knew that I wanted the course to include task-oriented breakout rooms and figure out ways to increase peer interactions in a way that was meaningfully tied to assessment. Sometimes when learners are put into breakout rooms, where there isn’t much follow up to the course or it’s not tied to evaluation, it can be difficult for students to stay on task.

For design objective two, I’m very conscious about academic integrity and trying to create situations that reduce the stress and temptation for learners to cheat especially with online learning post-Covid where students have access to different AI tools. I am also intentional about the reward  or “carrot” side to reduce cheating; there is research that says that community building in the classroom reduces opportunities for misconduct (Çolak, & Glendinning, 2021).

The third design objective theme was group work. Common issues can include frustration in group work where assigning grades can be difficult, and students get frustrated if they’re in groups where there is misalignment with their effort or their performance. That presented a challenge for me:  How might I have collaborative group activities, but with individual assessments?

What was your goal?

My goal was to design and deliver PROG8630 in an online learning environment which encompasses the design objectives so that we can continue to increase student engagement, learning, and career readiness.

And what were the actions you took?

Based on the existing course outline, I redesigned the 60% of the course evaluations  into a set of scaffolded carousel assessments.  Essentially, a scaffolded carousel assessment is a sequence of collaborative formative assessments with rotating project topics and artifacts. I deconstructed the existing course projects to add scaffolding so that each activity could rotate among the students.

 Each student was following the same path – the same series of tasks in the same order – but the topics rotated.  The course was modeled after activities which need to occur during the lifecycle of creating a dashboard. These seven assessments included a proposal, design specification, dashboard documentation, test report, dashboard demonstration, evaluation report, and a dashboard enhancement.

At the beginning of the semester, each of the students created a dashboard proposal. After submitting their proposal, the project topics rotated. For example, a student might have submitted a dashboard proposal related to Retail Industry, then worked on the design specifications for Finance Industry, then worked on a Sport Industry dashboard. Each of the topics were student created and rotated throughout the course. With each rotation, the topics and project work were shared with the next student.

So the students passed their work forward at each stage and started the next stage with the cumulative work of the previous students?

Yes. The students were a little bit resistant at the first switch since they were used to the traditional approach of working on one topic for the entire semester. However, I explained to them encouraging reminders about this authentic assessment such as:  “this is what you do at work. It’s not about the topics. This is a skill-based course – I’m teaching you how to get a project and move it along, mimicking the agile work environment, doing things in stages.”

The students really liked it as they worked with a variety of people for a short period of time and without sharing a grade. It was also helpful as I got to tell them, “Sometimes you’re going to get a great spec. Sometimes you won’t. That’s okay; let’s work together and figure it out.”

As students rotated their topics for the next assessment in the carousel rotation were they allowed to adjust or improve?

Yes, I told my students regularly “pivoting is part of the process” and the students appreciated that. If they felt stuck or unsure, we would meet one-on-one and adjust as necessary.

There were some people that were hesitant about it initially, but over time I didn’t get student complaints because it was very much like we’re in this together, we’re learning together.

But they didn’t actually have to show their grade to the next group?

No. Students knew from the beginning of the semester that the project work would rotate, so they were aware that their activities would be shared but their grades and feedback were not.  

What were some of the impacts you saw?

From the incidental student feedback that was collected at the end of the semester, two key themes were increased problem solving and creativity and increased engagement. The students identified that they had increased learning opportunities by being confronted with more situations. Students shared that they had an increased capacity to overcome obstacles and that that they could train themselves to be “ready for any project”.  With regard to engagement, students gave positive feedback that they’re able to get in touch with more people, expand their network, and that throughout this course they must connect with peers.

Students also identified challenges they had in the course, such as the amount of time to understand the new topics as well as the challenge of scheduling meetings and communicating frequently with peers. One way I attempted to reduce this was through regular in-class project time to reduce the amount of student meeting coordination outside of the classroom.

How did you decide to submit this?

 I regularly check the Faculty Learning Hub and the weekly Teaching and Learning newsletter for interesting workshops or tidbits that I can use. And so, when I saw the award, I was interested in the opportunity to get feedback on what I’ve been doing in the classroom, and to share something exciting. Additionally, while preparing for the submission, I gained confidence to submit a proposal about this topic for the McMaster Teaching & Learning 2022 Conference where I presented a poster about this teaching practice in December 2022. 

Sharing with Colleagues

The Scaffolded Carousel Assessment model does involve complexity and takes work to implement. The concept may not translate to other courses, but Amanda found the benefits worth the effort. If you’d like to learn more about Amanda’s project, here are visuals she provided to capture the process and the learning. (There is a link to the accessible PPT version below the photos).

Selected References:

1. Morris, M. E., Kuehn, K. S., Brown, J., Nurius, P. S., Zhang, H., Sefidgar, Y. S., … & Mankoff, J. C. (2021). College from home during COVID-19: A mixed-methods study of heterogeneous experiences. PloS one, 16(6), e0251580.
2. Ndovela, S., & Marimuthu, M. (2022). Prevalence of online cheating during the COVID-19 pandemic. In Academic Voices (pp. 443-455). Chandos Publishing.
3. Holden, O. L., Norris, M. E., & Kuhlmeier, V. A. (2021). Academic integrity in online assessment: A research review. In Frontiers in Education (p. 258). Frontiers.
4. Çolak, M. K., & Glendinning, I. Embracing Community-building in Online Classes to Promote Academic Integrity: A Case Study. In European Conference on Academic Integrity and Plagiarism 2021 (p. 41).
5. LaBeouf, J. P., Griffith, J. C., & Roberts, D. L. (2016). Faculty and Student Issues with Group Work: What is Problematic with College Group Assignments and Why?. Journal of Education and Human Development, 5(1).
6. Marks, M. B., & O’Connor, A. H. (2013). Understanding Students’ Attitudes About Group Work: What Does This Suggest for Instructors of Business?. Journal Of Education For Business, 88(3), 147-158.