Using Turnitin’s Generative Artificial Intelligence (GenAI) Detection Tool and the Process Guide for Navigating Potential Academic Offences

Reading Time: 4 minutes

Generative AI (genAI) has brought new challenges, opportunities, and questions in terms of the assessment of student work. Large Language Models (LLMs), the technology that powers ChatGPT and Microsoft Copilot, respond to prompts to produce human-like text within seconds.

The college continues to update its guidelines on genAI use.

Students may use genAI to create assessments and submit this work as their own. Students may also choose to use genAI as a helpful cognitive offloading tool and assistant to learning, and you may choose to define and condone such uses in your courses.

Unauthorized cases of genAI use have led to questions and penalties under Conestoga’s Academic Integrity Procedure, but we can expect increased adoption of genAI tools, especially as such tools become more accessible and embedded in higher education, as is the case with Conestoga’s Copilot.

Steps for Use of the Turnitin AI Detection Tool

Turnitin has developed an Artificial Intelligence (AI) detection tool that has been added to eConestoga. The tool has been trained to identify outputs from Large Language Models (LLMs).

1. Advise Students you are Using AI Detection.

• As of Spring 2023, a notice appears on all course outlines indicating that faculty will tell students what detection software is being used for each assessment.
• For example, you would indicate in writing that Turnitin’s genAI detection tool and Similarity Report or Respondus Monitor are being used and for which assignments or tests. Without such notice, a student may at some point appeal.

Note: Students may not opt out of Respondus LockDown Browser, only Respondus Monitor. 

• Without clear communication on genAI use in assessments (see #2 below), students may claim that you did not directly forbid the use of genAI. Respond that it has always been an expectation that all submitted work must be their original content or they must cite sources or collaborators.
• Do not submit student work to the Turnitin detection software without students’ knowledge.
  • For example, do not submit discussion posts to the Turnitin detection software without students being aware you are doing so. If you wish to check discussion posts for genAI, set up an Assignment drop box with Turnitin enabled and ask students to both post to the discussion board and submit their post as a Word document to the drop box.
• Do not submit student work to any external tools, such as GPTZero. Student work belongs to the student under Canadian Copyright law and serious privacy concerns exist with this practice.

2. Indicate that AI Can or Can’t be Used on Assignments.

• See our Hub post on “Generative Artificial Intelligence (GenAI) Assessment Statements for Students” for sample statements to use on your assignment descriptions.

3. Note the AI Writing Report on Turnitin, but Approach the Score with Caution.

• The AI Writing Report will give a percentage that indicates the amount of text that Turnitin’s AI detection tool has flagged as potentially genAI-generated. An AI Writing Report is generated for each assignment submission on eConestoga.
  • The AI Writing Report will flag text generated by Copilot, so use extra caution: students may be using Copilot in authorized ways. Clarify acceptable uses with students regarding Copilot well in advance of the assignment deadline.

• The AI detection percentage must be considered only one data point in the marking process rather than evidence of an academic offence. The Turnitin detection tool shows a false positive rate of 1%. For example, the detection tool may falsely flag repetitive writing as suspicious. This video from Turnitin explains false positives. Turnitin’s website provides additional information on their AI detection software.

• The percentage reflects only that portion of the text that the detection tool was able to analyze. Turnitin’s detector analyzes long-form English prose. It excludes short-form answers, lists, bulleted points, code, outlines, poetry, and any text that does not appear in standard grammatical sentences.

• Students cannot currently see the AI detection score. You will need to inform them of the percentage if proceeding to use this data point in an offence.

4. Use Other Data Points. 

Use additional data points to determine if a student has submitted text created using genAI before assigning an academic offence.

Possible indicators of genAI use include:

• A very low Turnitin Originality score combined with a high AI detection percentage.
  • “High” means anything over 80%, but again approach that score with caution. 
• Missing, inaccurate or fabricated citations and references.
• Significant departure from the requirements of the assignment (e.g., a lengthy answer that is not required, irrelevant responses or explanations).
• Student demonstrating skill or learning far beyond a level previously demonstrated.
• Uncharacteristically strong submission for the course.
• Extreme, formulaic specificity of response to the question/prompt.
• Highly specific, procedural organization of ideas that directly maps on to prompt (i.e., if the assignment/test question asks students to “Describe X and then explain Y’s importance to Z” a generative AI response will do exactly that, in that order).
• Text copied and pasted from ChatGPT, which can include phrases such as “As an AI language model . . .” or the disclaimer from the app’s footer: “ChatGPT Mar 23 Version. Free Research Preview. ChatGPT may produce inaccurate information about people, places, or facts.”

5. Proceed to a Concern Notice When Multiple Data Points are Present.

1. As per the college’s Academic Offences Procedure, communicate to the student via email that you have some concerns about their assignment. Explain the concerns/evidence you have found suggesting possible misconduct and ask the student to respond explaining what you are seeing.Be transparent with the student about what evidence you have collected. Do not rely on the Turnitin score alone.
2. Connect with the Manager, Academic Integrity, Cory Scurr (cscurr@conestogac.on.ca) to obtain a copy of the Process Guide for navigating potential misconduct. In addition to the above-mentioned suggestions, the Process Guide provides faculty with a “Rubric” to help facilitate the conversation.
3. If meeting for a conversation with the student to ascertain attainment of the associated course outcomes is required:
   • Explain the purpose of the meeting in writing, detail what will occur, and schedule a time at least two days in advance.
   • Account for any accommodations.
   • Do not refuse a student request to have a support person present.
4. During this meeting, you might ask the student to complete a small selection of the following:
   • Define specific concepts and terms from the text.
   • Answer key questions related to the content of their submission.
   • Elaborate on an example or generalization provided in the assignment.
   • Reflect on how their understanding of the topic has changed over time.
   • Share preparatory or draft work done toward the assignment (in line with process discussion).
   • Describe how they came to a particular view, opinion, or conclusion.
   • Discuss their process of preparing the assignment.
   • Explain their selection of sources or provide links to selected sources.
5. The purpose of this conversation should be to gauge if the student’s written work reasonably reflects their understanding, knowledge, and self-described process for completing the assignment. Substantial discrepancies between verbal and written evidence of learning may indicate use of genAI in the written submission, although this interpretation should be approached with caution. Some students may be stronger speakers than writers, and vice versa. Throughout, it is important to take notes and document all discussions with the student and approaches used in gathering evidence. Consult the Process Guide (available through Cory Scurr) for further information.
6. With sufficient and robust evidence, faculty may choose to file an academic offence through the Academic Offences Portal via the Employee Portal. Connect with the Academic Integrity Office for support and clarification in filing offences.
   • The most likely selection for such an offence is Unauthorized Aids and Assistance (Cheating) or Copying from Others or Self Copying.

Glossary

Turnitin AI Detection Percentage: Added in April 2023 as a feature of the Similarity Report, this provides an overall percentage of the submitted text that was likely generated by genAI. The percentage reflects only the text that the detection tool can analyze – paragraphs of English in complete sentences – and excludes bullets, lists, code, outlines, and other types of text. Turnitin provides further information on how their detection software works.

Turnitin Originality Report: Uses metadata and forensic language analysis to distinguish between the student’s own work and other sources but will not highlight genAI-generated text.

Turnitin Similarity Report: This report summarizes matching or very similar text present in another piece of writing. A similarity score is assigned to the submission, expressed as a percentage. Turnitin offers an overview to assist in interpreting Similarity Reports.