‘Frenemy’ of progress? Investigation of the disruptive impacts of generative pre-trained transformers (GPT) on learning and assessment in higher education
Date
2023-08
Authors
Kolade, Oluwaseun
Owoseni, Adebowale
Egbetokun, Abiodun
Journal Title
Journal ISSN
ISSN
DOI
Volume Title
Publisher
Peer reviewed
Yes
Abstract
ChatGPT, a state-of-the-art chatbot built upon Open AI’s generative pre-trained transformer (GPT-3), has generated a major public interest and caused quite a stir in the higher education sector, where reactions have ranged from excitement to consternation. With approximately 175 billion parameters at its command, GPT-3 is one of the largest and most powerful natural language processing AI models available, with vast and versatile capabilities surpassing previous chatbot models. We conducted a quasi-experiment in which we deployed ChatGPT to generate academic essays in response to a typical assessment brief, and then subjected the essays to plagiarism checks. In addition, Chat GPT was instructed to generate contents in various formats, including editorial and poetry, and the output was subjected to summary thematic analysis. The results of the quasi-experiment show that ChatGPT is able to generate highly original, and high quality, contents from distinct individual accounts in response to the same assessment brief. However, it is unable to generate multiple original contents from the same account, and it struggled with referencing. The discussion highlights the need for higher education providers to rethink their approach to assessment, in response to disruption precipitated by artificial intelligence. Thus, following the discussion of empirical data, we propose a new conceptual framework for AI-assisted assessment for lifelong learning, in which the parameters of assessment extend beyond knowledge (know what) testing, to competence (know how) assessment and performance (show how) evaluation.
Description
The paper was presented in the Knowledge and Learning Track at the 2023 British Academy of Management Conference (BAM2023) in-person, which took place from the 1st September – 6th September and was hosted by the University of Sussex Business School.
Keywords
Research Subject Categories::INTERDISCIPLINARY RESEARCH AREAS, ChatGPT, assessment, higher education
Citation
Kolade, O., Owoseni, A. and Egbetokun, A. (2023) ‘Frenemy’ of progress? Investigation of the disruptive impacts of generative pre-trained transformers (GPT) on learning and assessment in higher education.
Research Institute
Centre for Enterprise and Innovation (CEI)