Preprint
Large Language Models

ChatGPT as an Educational Tool: Opportunities, Challenges, and Recommendations for Communication, Business Writing, and Composition Courses

Mohammad Awad AlAfnan(American University of the Middle East), Samira Dishari(American University of the Middle East), Marina Jovic(American University of the Middle East), Koba Lomidze(American University of the Middle East)
March 6, 2023Journal of Artificial Intelligence and Technology401 citations
PDFDOI

401

Citations

0

Influential Citations

Journal of Artificial Intelligence and Technology

Venue

2023

Year

Abstract

This empirical study examines ChatGPT as an educational and learning tool. It investigates the opportunities and chal-lenges that ChatGPT provides to the students and instruc-tors of communication, business writing, and composition courses. It also strives to provide recommendations. After conducting 30 theory-based and application-based ChatGPT tests, it is found that ChatGPT has the potential of replacing search engines as it provides accurate and relia-ble input to students. For opportunities, the study found that ChatGPT provides a platform for students to seek an-swers to theory-based questions and generate ideas for ap-plication-based questions. It also provides a platform for in-structors to integrate technology in classrooms and conduct workshops to discuss and evaluate generated responses. For challenges, the study found that ChatGPT, if unethically used by students, may lead to human unintelligence and un-learning. This may also present a challenge to instructors as the use of ChatGPT negatively affects their ability to dif-ferentiate between meticulous and automaton-dependent students, on the one hand, and measure the achievement of learning outcomes, on the other hand. Based on the out-come of the analysis, this study recommends communica-tion, business writing, and composition instructors to (1) re-frain from making theory-based questions as take-home as-sessments, (2) provide communication and business writing students with detailed case-based and scenario-based as-sessment tasks that call for personalized answers utilizing critical, creative, and imaginative thinking incorporating lec-tures and textbook material, (3) enforce submitting all take-home assessments on plagiarism detection software, espe-cially for composition courses, and (4) integrate ChatGPT generated responses in classes as examples to be discussed in workshops. Remarkably, this study found that ChatGPT skillfully paraphrases regenerated responses in a way that is not detected by similarity detection software. To maintain their effectiveness, similarity detection software providers need to upgrade their software to avoid such incidents from slipping unnoticed.

Analysis

Why This Paper Matters

As large language models like ChatGPT become increasingly accessible, their role in education demands careful examination. This paper addresses a critical gap by empirically testing ChatGPT's capabilities and limitations in communication, business writing, and composition courses—areas where writing and critical thinking are paramount. With 401 citations, it has quickly become a reference point for educators grappling with AI's presence in the classroom.

The study's significance lies in its balanced perspective: it neither demonizes nor glorifies ChatGPT. Instead, it systematically explores opportunities (e.g., personalized learning support, idea generation) and challenges (e.g., academic dishonesty, skill atrophy). By offering concrete recommendations, it bridges the gap between theoretical concerns and practical pedagogy.

Moreover, the finding that ChatGPT can paraphrase responses to evade plagiarism detection software raises urgent questions about academic integrity infrastructure. This has implications beyond individual courses, affecting how institutions design assessments and uphold standards in an AI-augmented world.

Technical Contributions

  • Empirical Testing Framework: The authors designed 30 tests (theory-based and application-based) to evaluate ChatGPT's responses, providing a replicable method for assessing LLMs in educational contexts.
  • Opportunities Identification: Demonstrated ChatGPT's potential as a reliable answer engine for theory questions and a creativity aid for application questions, suggesting a shift from search engines to conversational AI.
  • Challenge Analysis: Highlighted risks of "human unintelligence and un-learning" and instructors' difficulty in distinguishing authentic student work from AI-generated content.
  • Actionable Recommendations: Proposed four pedagogical strategies: avoid take-home theory questions, use case-based assessments, enforce plagiarism detection, and integrate ChatGPT outputs as discussion material.
  • Paraphrasing Evasion: Revealed that ChatGPT can regenerate responses to bypass similarity detection software, exposing a vulnerability in current plagiarism tools.

Results

The study found that ChatGPT provided accurate and reliable input across the 30 tests, positioning it as a potential replacement for search engines in educational settings. For theory-based questions, it offered direct answers; for application-based questions, it generated relevant ideas. However, the model also skillfully paraphrased regenerated responses such that similarity detection software failed to flag them. This dual finding—reliability and evasion—underscores the need for both pedagogical adaptation and technological upgrades in detection tools. No quantitative metrics beyond test counts were reported.

Significance

This paper has significant implications for AI in education. It urges instructors to redesign assessments toward personalized, critical-thinking tasks that AI cannot easily replicate, and it calls for plagiarism detection software to evolve. The concept of "human unintelligence" warns of cognitive offloading risks. As LLMs proliferate, this work informs policy and practice, emphasizing ethical use and the preservation of learning outcomes. It also signals to AI developers the importance of transparency and watermarking to support academic integrity.

chatgpteducationlarge-language-modelsacademic-integrityplagiarism-detectionbusiness-writingcomposition-studies

Related Papers