Enhancing Academic Integrity with AI Detection in Turnitin

Turnitin AI Detection Canvas & Teaching Technology Monthly Meeting Gemma Sinclair Product Manager June 2023

Turnitin AI Detection We are now able to test the new AI Detection Score in Turnitin. Two test courses have been setup in testing area in Canvas with a separate Turnitin external tool with AI detection enabled. We have done some basic testing with sample documents and looking to discuss initial findings with Academic Integrity Community of Practice next week. Happy to add more staff to the course for testing please let me know if you d like to be added. Less than 300 words AI generated text Due to exams, we will continue testing during inter-semester break. 2

Turnitin AI Detection Similarity Score Essay generated by ChatGPT and submitted to Turnitin. Similarity 2% AI Detection score 75% AI Detection Score 3

Turnitin AI Detection AI Detection Score AI report highlights the text segments that Turnitin model predicts were written by AI Link to FAQ page to provide additional information Turnitin support resources 4

Update from Turnitin CPO Annie Chechitelli, Chief Product Officer at Turnitin Instructors and administrators cite false positives as a main wrote a blog post about key learnings from the AI concern of AI writing detection writing detection after the release in April. In summary, these updates are being released to address our customers false positive concerns: displaying an asterisk in the indicator for documents with less 3.5% of submissions contain more than 80% AI-written text. than 20% AI writing detected, As of May 14, 2023, out of the 38.5 million submissions we raising the minimum word requirement from 150 to 300, and have processed for AI writing, 9.6% report over 20% of AI adjusting how we aggregate sentences at the beginning and writing and 3.5% report between 80% and 100% of AI writing. ending of the document. Link: https://www.turnitin.com/blog/ai-writing-detection-update-from-turnitins-chief-product-officer 5

Turnitin AI Detection AI Detection Score AI detection scores below 20% are less accurate and will show an asterisk next to the score 6

Turnitin AI Detection Resources from Turnitin Approaching a student regarding potential AI misuse: This guide supports honest, open dialogue with students regarding their work without either party becoming overly defensive during what can be a difficult interaction. Discussion starters for tough conversations about AI: This guide helps educators focus on growth in the work that has been submitted and facilitate productive conferences. AI conversations: Handling false positives for educators: This guide shares strategies educators can consider before and after submissions when discovering a false positive. AI conversations: Handling false positives for students: This guide for students shares strategies they can consider before and after submissions when confronted with a false positive. Ethical AI use checklist for students: This checklist for students provides guidance during all phases of work and suggests ways for them to make decisions that support integrity and align with educator guidelines. 7

AI Detection - next steps 1. Test AI detection tool this is also being done with AUSA 2. Review findings with Academic Integrity CoP and Learning Environments Subcommittee ahead of TLQC in August 3. Recommendation to Education leadership and Assoc Deans L&T for use in Semester 2, 2023 4. Write guidelines and communications for teaching staff for using the detection score (if enabled) 8

Gen-AI in Teaching & Learning Mon, 12 Jun 2023 1:00 PM - 2:00 PM NZST What questions do you have about Gen-AI in teaching and learning at the University of Auckland? Questions may range from incorporating GenAI into your course, to responding to cases of use of GenAI to facilitate academic misconduct. The following panelists will be available to answer your questions: Dr Gayle Morris, Director of Learning and Teaching Dr Jason Stephens, Education and Social Work Dr Paul Denny, Computer Science Varsha Ravi, Education Vice President, AUSA Dr Steve Leichtweis, Head of eLearning Group Gemma Sinclair, Product Owner Turnitin Rebecca White, Academic Quality Office We will also be hearing from Dr Alex Sims and Dr Udayangi Muthupoltotage about best practice for assessment design in the face of this changing landscape. This event will be recorded. https://www.eventbrite.co.nz/e/gen-ai-in-teaching-and-learning-uoa-tickets-648504342947 9

Thank you