UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF ENGLISH LANGUAGE TEACHER EDUCATION SCIENTIFIC RESEARCH REPORT Benefits and challenges of ChatGPT in learning writing skill
INTRODUCTION 21222222121 121212111112211120 20111111 re, 6
Research aims and research questIO'S - 0 2222111221211 2221112111151 1 111 ke 8 li án
This study aimed to investigate the perceptions of FELTE students at ULIS regarding the use of ChatGPT as a tool to enhance their writing abilities, and to assess its overall effectiveness in improving writing skills, focusing on key research questions that guide the investigation.
(a) What are students' perceptions of the benefits/challenges of ChatGPT to their writing?
(b) At which stage of the writing process are students more likely to use ChatGPT?
The study's findings will empower students to gain a deeper understanding of ChatGPT, including its benefits, challenges, and potential applications, ultimately enabling them to harness technology to enhance their writing skills By equipping English as a Foreign Language teachers with the necessary skills and knowledge, they can effectively integrate ChatGPT into their teaching practices, particularly in writing instruction Furthermore, the study's insights into students' behavior when using ChatGPT for writing will inform the development of guidelines and course policies that address plagiarism and academic integrity concerns, promoting a more effective and responsible use of this technology.
Besides practical significance as previously mentioned, our research may also contribute to the continuous development and improvement of Al-based writing tools by highlighting areas for enhancement based on feedback from students Additionally, we aspire to establish a platform for further research in the field of artificial intelligence in education in general and writing in particular.
LITERATURE REVIEW SH H012 1111k, 9
METHODOLOGY Q.2 2.12 12101211121111101 20111 11 1e, 17 3.1 Research approach and research đesign .- 1c 22 2211211121152 21 111211212211 k2 17
Research population and research sarmpÌe - 2 222122112112 1221218115221 18 3.4 Data collection con
The study's participants consisted of undergraduate students from the Faculty of Language Teacher Education (FELTE) at the University of Languages and International Studies, all of whom were pursuing degrees in English Language Teaching (ELT) and were expected to possess a strong foundation in English language skills A voluntary sampling method was employed to collect data, with the survey distributed via email to all FELTE students and posted on relevant Facebook groups Efforts were made to ensure a representative sample by soliciting responses from students across different cohorts.
In the end, the research got responses from 128 participants Demographic information of the participants is presented below
Table 3.1: Demographic information of the participants
Year of study in ULIS
Here is a rewritten paragraph that captures the essence of the original text, optimized for SEO:"Notably, a substantial 16.1% of participants achieved English proficiency over a decade or more, while a mere 0.8% had been exposed to the language for less than 2 years or beyond 16 years, highlighting a significant disparity in language acquisition timelines among the subjects."
The data collection method employed in this study involved a questionnaire survey, which was utilized to gather relevant data Specifically, the study's data was collected from FELTE students through an online questionnaire hosted on Google Forms, providing a convenient and accessible platform for respondents to participate.
A questionnaire is a written method of data collection used to gather information from respondents Leveraging online platforms like Google Forms facilitates the collection of data from a wider and more diverse population of students, overcoming geographical limitations and increasing the representativeness of the findings Furthermore, using Google Forms streamlines data management by automatically recording, sorting, exporting, and storing responses.
& Harinarayana, 2016) It could simplify the data and analysis process, save time, and reduce potential errors (Brigham, 2014)
The questionnaire included four main parts The first part consisted of the introduction of the survey and participants’ information in the form of short answer questions In this first section, the purposes and scope of the research were also explained to the participants, and their informed consent would be obtained The participants were assured that their identities would remain confidential, and any information shared would be anonymized to protect their privacy
Here is a rewritten paragraph that captures the essence of the original text, optimized for SEO:The second part of the questionnaire focused on gathering insights into participants' usage and perceptions of ChatGPT's effectiveness in enhancing their English writing skills This section employed a range of question types, including binary, multiple-choice, open-ended, and Likert scale questions, to assess participants' engagement with ChatGPT, identify the stages at which they utilized it during the writing process, and evaluate its overall effectiveness By using a mix of question formats, the survey aimed to provide a comprehensive understanding of how ChatGPT supports English writing skills development, including the specific stages where it is most useful and participants' levels of agreement with its effectiveness.
The study's third part evaluated participants' attitudes towards using ChatGPT in writing through a five-point Likert scale, assessing their level of agreement or disagreement with statements covering various aspects of the writing process This included task analysis, researching, brainstorming, vocabulary suggestions, grammar correction, and concerns about ethics and motivation The survey examined participants' perceptions of ChatGPT's assistance in the prewriting, writing, and post-writing stages, as well as its impact on critical and independent thinking skills Key areas of focus included the tool's ability to support task analysis, researching, and brainstorming, provide vocabulary suggestions and grammar correction, and enhance sentence structure and suggest alternatives or synonyms Additionally, the study explored participants' views on the motivational aspects of using ChatGPT for writing skills development and their concerns about ethics, privacy intrusion, and the impact on critical thinking and language ability.
22 The last part of the questionnaire was about solutions for students to make use of ChatGPT effectively Several solutions were presented as a list of options for participants to select one or more Furthermore, the "Others" option allowed participants to suggest any additional solutions not covered by the given options This provides flexibility for participants to express their unique perspectives and propose solutions that they believe are necessary for effective use of ChatGPT in writing
Details of the questionnaire could be found in the Appendix.
Method of data anaÌy§15 - - L L2 102010120 1112111211 11111111111 51 1111151 1kg khay 20
Data collected from the questionnaires were analyzed quantitatively using SPSS software (Statistical Package for the Social Sciences) (Version 22) Specifically, descriptive analyses were attempted Mean scores (M) and standard deviation (SD) of the scales, sub-scales, and indicators (question item) were calculated, compared and contrasted Cronbach's alpha values of the (sub-)scales were also calculated prior to use of the scales in analysis In interpreting the Cronbach's alpha values, the research applied the general rule of thumb that a Cronbach’s alpha of 70 and above is good, 80 and above is better, and 90 and above is best (George & Mallery, 2003).
FINDINGS AND DISCUSSION ke 21 "sẽ nan
DISCUSSION
This research aimed to explore FELTE students’ perspectives on the use of ChatGPT in learning writing Overall, students largely acknowledged the benefits of ChatGPT to improve their writing skills in all three stages: pre-writing, writing, post-writing The most popularly recognized benefits of ChatGPT include its ability to save writing time and its flexibility Such findings align with Karthikeyan (2023) and Woo, Guo and Susanto (2023) who claim that ChatGPT offers immediate feedback, reducing the time students wait for teachers
Upon examining the various stages of the writing process, it was evident that students found ChatGPT most beneficial during task analysis, where it helped them break down steps to approach a topic, identify the task's target, simplify requirements, and explain specialized terms, a finding that aligns with the research of Frackiewicz.
(2023), which suggests that ChatGPT excels in simplifying task requirements However, it is worth noting that a majority of studies assert that students tend to benefit from auto-generated prompts rather than task analysis (Abdullayeva and Musayeva, 2023; Liu, 2023)
ChatGPT demonstrated its potential in brainstorming by generating numerous ideas from diverse aspects of a topic, aligning with the findings of Vicente et al (2023), Harunasari (2022), and Dergaa et al (2023) However, FELTE students did not perceive ChatGPT as capable of reverse prompting, contradicting the insights of Chang, Lin, and Hajian (2023) In terms of researching, FELTE students found the information provided by ChatGPT to be general and inconsistent in quality, raising concerns about potential biases and inaccuracies, as also noted by Baskara (2023) and Beck and Levine (2023).
Research has shown that auto-generated prompts supported by evidence and diverse vocabulary are most effective during the writing stage, as noted by Yan (2023) and Woo et al (2023) Additionally, studies have found that ChatGPT can provide coherent samples with clear structure, as highlighted by Fitria (2023) However, FELTE students have expressed concerns about ChatGPT's ability to predict potential counter-arguments, a notion that has not been extensively explored in previous studies, although Yan (2023) suggested that detecting counter-arguments may be a challenge for ChatGPT, particularly with complex tasks.
Concerning the category of post-writing stage, it is commonly believed that ChatGPT may excel in identifying basic grammar and spelling errors (Garaady and Mayhoob,
Here is a rewritten paragraph based on the provided content:According to recent studies, ChatGPT's capabilities in suggesting alternative expressions, synonyms, and antonyms align with the findings of this research (Abdullayeva and Musayeva, 2023) However, some FELTE students expressed disagreement, citing that ChatGPT often falls short in providing detailed justifications for its feedback, contradicting the viewpoints of Abdullayeva and Musayeva (2023).
In terms of learner's motivation and ethical issues Most student participants in the current study expressed concerns about plagiarism and copyright infringement, which 1s In line with the majority of previous studies such as Karthikeyan (2023) and Liu
Research on ChatGPT has raised several concerns, with students primarily focusing on its potential to violate academic standards However, issues such as privacy intrusion and data secrecy have been relatively overlooked Notably, ChatGPT appears to lack the ability to motivate students, which aligns with some studies suggesting it can depreciate learners' effort Furthermore, the tool has been found to have limited contribution to developing students' critical, independent, and creative thinking skills, highlighting its limitations in promoting deeper learning outcomes.
33 explained by the overreliance on technology to operate writing tasks and the subjectivity of information generated by the tool
To sum up, these differing views emphasize the need for a clearer understanding of ChatGPT's capabilities, revealing its potential gaps between learners’ expectations and the tool's performance efficiency in writing tasks.