Faculty Research Highlight
CommServ Institute of AI and Professional Studies is pleased to highlight faculty research examining an important question in the age of Generative AI: how effectively can AI-content detection tools identify AI-generated and AI-assisted writing in higher education?
Evaluation of AI Content Generation Tools for Verification of Academic Integrity in Higher Education
Authors: Muhammad Bilal Saqib and Saba Zia
Journal: Journal of Applied Research in Higher Education
Volume: 17
Issue: 4
Pages: 1430–1440
DOI: 10.1108/JARHE-10-2023-0470
Publisher: Emerald Publishing
Read the original research article on Emerald Publishing
About the Research
The rapid adoption of Generative AI has created new opportunities for writing, learning and research, while also creating new challenges for educators and institutions concerned with academic integrity. This research evaluates AI-content detection tools and examines their performance across different types of AI-assisted writing.
The study evaluated eight AI detection engines using four categories of content: original human-generated writing, AI-paraphrased content, fully AI-generated content, and AI-generated content incorporating contextual information.
Key Findings
Original Human-Generated Content
The cumulative mean for original human-generated material was 1.71% AI content in the study's testing conditions.
Fully AI-Generated Content
For content generated entirely by AI, the cumulative mean was 76.85% AI content.
AI-Paraphrased Content
For AI-paraphrased material, the cumulative mean was 39.42% AI content.
AI-Generated Content with Contextual Information
For AI-generated material produced using contextual information, the cumulative mean was 60.1% AI content.
Implications for Academic Integrity
The findings demonstrate that AI-content detection performance can vary according to how content is produced. This is particularly relevant in higher education, where AI use may range from fully generated text to AI-assisted or paraphrased writing.
The research contributes to the discussion around responsible use of AI-detection technologies and the interpretation of their results in academic settings.
Faculty Research at CommServ
At CommServ Institute of AI and Professional Studies, we believe professional AI education should extend beyond learning how to use AI tools. Understanding AI systems, evaluating their limitations, and examining their impact on education and professional practice are important parts of responsible AI learning.
Faculty research helps connect practical technology education with research, critical thinking and responsible use of Artificial Intelligence.
Publication Details
Saqib, M. B., & Zia, S. Evaluation of AI content generation tools for verification of academic integrity in higher education. Journal of Applied Research in Higher Education, 17(4), 1430–1440.
DOI: 10.1108/JARHE-10-2023-0470
Read the Original Research
Research focus: Artificial Intelligence, Generative AI, AI content detection, academic integrity and higher education.