INTRODUCTION
Statement of research problems and rationale for the study
Providing feedback to students is crucial in formative assessment, as it plays a significant role in enhancing their skills Constructive and informative feedback, as highlighted in Kerr's (2017) book "Giving Feedback on Speaking," has been shown to effectively improve language learning outcomes.
However, giving comments on students' real-time interpreting is stressful
As an interpreter trainer, I seek a more effective tool for providing feedback in interpreting classes, moving beyond rushed, handwritten notes My goal is to enhance the feed-forward process, ultimately improving students' interpreting performance more efficiently.
To date, there is a consensus among social scientists that computer-assisted feedback is likely to bring about a variety of advantages Smith and Harris
Advanced technology plays a crucial role in enhancing the training experience for interpreter trainers by offering a superior alternative to traditional physical media, as noted in 2014 Smith and Harris highlighted specific advancements, such as time-coded feedback and commentary, which facilitate improved communication between students and interpreter trainers through innovative tools like GoREACT.
Dodd and Kusaka (2014) explored the benefits of an innovative app in their institution, highlighting its ability to record and synchronize spoken and written texts into digital files accessible on computers and smart devices This technology enables teachers to capture students' speaking activities along with feedback notes, which can be sent to students post-lesson Students can easily replay these digital files and view their teacher's feedback at the relevant moments, helping them understand the context of the feedback The combination of visual and auditory elements is particularly appealing to diverse learners and supports teachers in providing context-rich feedback (Dodd & Kusaka, 2014, p.44).
Simultaneous interpreting presents a significant challenge for new interpreters and students due to the overwhelming amount of information they must process (Seeber, 2011) The intense time pressure and cognitive stress in advanced interpreting courses, which include both consecutive and simultaneous modes, often lead students to forget key aspects of their performance immediately after interpreting This makes it difficult for them to correlate teacher feedback with their spoken output, particularly regarding mistakes in pronunciation, grammar, and word choice Consequently, decontextualized feedback has become a persistent issue in interpreting courses However, with the use of Audionote, I can easily replay recordings and provide students with contextualized feedback at specific moments in their speaking activities, thereby enhancing their learning experience.
As an interpreter trainer, I find the recent findings particularly intriguing, especially given the challenges I've faced in providing effective feedback to my students during interpreting lessons However, there is a notable lack of research on the use of Audionote for offering feedback to interpreter trainees.
Those findings and arguments are the main reasons triggering my research study.
Educational context and participants
1.2.1 Setting of the chosen class
The study was conducted at the Interpreter and Translator Training Division of a foreign language university in Vietnam, involving a class of twenty senior students majoring in English translation and interpreting Each participant had previously completed at least one interpreting practice course, gaining experience in traditional feedback methods used by interpreter trainers This semester, they enrolled in their Advanced Interpreting course, marking their second interpreting course within the curriculum, during which the intervention phase occurred.
The classroom is well-equipped with essential teaching facilities, including a projector, speaker, two interpreting booths, an interpreting toolkit, and reliable internet access Both teachers and students have access to smartphones and laptops, enabling them to record audio files and access basic digital documents.
1.2.2 Identification of the problems in the class
Large class sizes, such as 20 students, pose significant challenges for teachers in providing individual practice opportunities during lessons Typically, in a three-period session, only around 10 students receive the chance to engage and obtain detailed feedback Consequently, this limitation hampers student participation and reduces their opportunities to express themselves in class.
Providing feedback on students' real-time interpreting can be quite stressful for interpreter trainers The hurried, handwritten notes often fail to meet the necessary standards of quality Recognizing the significance of formative feedback, it is crucial to deliver more comprehensive and effective evaluations to support students' growth and development.
Student interpreters experience significant stress during advanced interpreting courses, making it challenging for them to remember their interpretations and reflect on teachers' feedback after performances This issue is compounded by the traditional feedback from teachers, which often lacks the necessary context for each comment.
Large class sizes significantly impede students' opportunities for in-class practice, while the lack of personalized, contextualized feedback on their interpreting performance further exacerbates the issue.
Aims and objectives of the study
This thesis explores the impact of Audionote technology on enhancing students' interpreting performance and their responses to the teacher's feedback methods using Notepad and Voice Recorder Additionally, it investigates potential challenges the teacher may face when implementing this technology in the classroom, along with possible solutions The research specifically addresses three key questions related to these themes.
Research question 1: What were the opinions of the students about the use of Audionote in the teacher’s feedback for the students’ classroom interpreting performances?
Research question 2: To what extent did the application of Audionote in the teacher’s feedback improve the students’ interpreting performances?
Research question 3: What were the difficulties of the teacher when applying Audionote in giving feedback for the students’ classroom interpreting performances (if any)?
The research questions were specifically designed to address the unique challenges faced in my interpreting class, without intending to generalize findings to other classes with varying contexts and student characteristics The study's results aim to provide targeted solutions for the identified issues within this particular class, laying the groundwork for future research on similar topics.
Scope of the study
This action research study only focuses on investigating the use of Audionote (Notepad and Voice Recorder) in giving feedback for the students‘ interpreting performance
In this article, I focused exclusively on Audionote as the intervention tool, which combines notepad and voice recorder functionalities Other similar applications with varying features will not be addressed in this discussion.
This research focuses specifically on the teacher's formative feedback within the classroom environment, and it does not address other types of feedback that may occur in different contexts.
This action research study focused on investigating the introductory level of Vietnamese-English simultaneous interpreting among students Consequently, it did not address the use of Audionote for providing feedback to student interpreters in different interpreting contexts or educational settings.
Finally, in this study, the term ―performance‖ was used instead of
The concept of "competence" emphasizes students' practical skills in interpreting over their theoretical knowledge Chomsky (1969) distinguished between "competence," which is the knowledge of a language, and "performance," the actual use of that language in real situations This distinction was later supported by Bachman (as cited in Brown, Malmkjòr & Williams, 2004), reinforcing the idea that competence is crucial for effective interpretation.
―unobserved, underlying knowledge‖ of the language, meanwhile ―performance the term ―performance‖ consistently throughout the paper instead of
Significance of the study
On completion, the study brought about certain benefits to interpreting students, interpreter trainers, researchers and the entire society in general who share the same interest in this topic
This research highlights the importance of interpreting students' perspectives on their trainers' feedback, emphasizing its significant impact on their learning process Despite limited previous studies in this area, understanding student reflections is essential for enhancing the educational experience The paper investigates student interpreters' views on trainers' feedback and its effects on their learning, aiming to offer valuable insights for interpreter trainers The findings are intended to guide educators in improving their teaching methods, ultimately fostering more effective support for students' skill development and learning outcomes.
Interpreter trainers face challenges in enhancing the quality of their in-class feedback due to a scarcity of studies on assessing and improving this feedback This action research offers a potential new method for trainers to implement in their classrooms, aiming to improve the overall quality of interpreting training.
In the third place, as for researchers who take interest in the same topic, they can refer to this paper as a source of updated and reliable information
This action research paper does not seek to generalize the effectiveness of Audionote across all interpreting classes; instead, it aims to address specific challenges within the selected class This focused approach serves as a foundation for future large-scale research.
Structure of the thesis
The study is structured into five key chapters: Chapter 1 introduces the research problem, rationale, aims, and scope; Chapter 2 offers a comprehensive literature review that establishes the theoretical framework and key concepts; Chapter 3 details the methodology, outlining the data collection and analysis processes; Chapter 4 presents the data analysis, findings, and discussion, highlighting the main research outcomes; and Chapter 5 concludes the study by summarizing the findings, acknowledging limitations, and proposing suggestions for future research.
LITERATURE REVIEW
Feedback
Feedback is a crucial element in education, as defined by Kerr (2017, p 2) as any information that students receive regarding their performance It can take the form of corrective feedback, which highlights errors, or non-corrective feedback, such as praise and encouragement Additionally, feedback may also pertain to the performance of peers, providing valuable insights for improvement.
―not only answer correctness, but other information such as precision, timeliness, learning guidance, motivational messages, lesson sequence advisement, critical comparisons, and learning focus‖ (Hoska, 1993)
Scholars have varying perspectives on the aims of formative feedback, but a consensus exists around its primary goal of enhancing students' knowledge and skills in relation to specific learning objectives Shute (2008) emphasizes that formative feedback should promote comprehension and problem-solving abilities, while Hattie and Timperley (2007) highlight its role in bridging the gap between current performance and desired standards According to Shute, feedback serves two key purposes: directive feedback, which identifies necessary corrections, and facilitative feedback, which provides suggestions to guide students in their revisions Ultimately, formative feedback is designed to support learners in achieving their educational goals.
The literature presents various methods for categorizing feedback, including formal versus informal, qualitative versus quantitative, and teacher versus peer feedback For the purposes of this study, the researcher will concentrate on two primary categorization methods that are most relevant.
Feedback can be categorized into qualitative and quantitative types, with qualitative feedback comprising textual comments and quantitative feedback represented by numerical marks (Hamer et al., 2015) Formative feedback focuses on assessing the accuracy and adequacy of a student's performance, addressing specific errors and misconceptions (Shute, 2008).
Feedback is categorized into three types: peer feedback, teacher feedback, and self-feedback External feedback from peers and teachers is crucial for enhancing students' skills, as it provides objective perspectives and constructive advice In contrast, self-feedback allows students to reflect on their own performance, identifying their strengths and weaknesses This reflection encourages learners to focus on specific areas for improvement and develop their skills effectively (Lee, 2018, p 154).
This research study defines feedback as the comments provided by interpreter trainers to their students during interpreting lessons, aimed at enhancing students' interpreting competence The focus is primarily on qualitative and formative feedback rather than quantitative metrics Throughout the interpreter training course examined in this study, the trainer occasionally incorporated both numerical scores and textual feedback in her evaluations Furthermore, the study emphasizes the importance of teachers' roles in the feedback process.
The significance of giving feedback in both English language teaching and interpreter training has been repeatedly asserted by many researchers of the field
In "Giving Feedback on Speaking," Kerr (2017) emphasizes that feedback significantly enhances language learning, supported by evidence Hattie and Timperley (2007) also highlight feedback as a crucial factor influencing learning and achievement.
In 2018, the significance of feedback in learning was highlighted, particularly in interpreter training, where it serves as a vital tool for correcting mistakes and enhancing language skills Despite its importance, research on feedback in interpreting courses is limited, and there is a need to explore methods for improving feedback quality for trainers Furthermore, with rapid technological advancements, it is essential to investigate how these technologies can aid interpreter trainers in delivering more effective feedback to student interpreters.
Feedback can be delivered either in person or through technology, allowing teachers to enhance student performance Educators can take notes during the students' activities and provide feedback later, or they can offer written or verbal responses after reviewing recordings of student performances (Lee, 2018).
Research highlights the numerous benefits of technology-assisted feedback, suggesting that "technology-enhanced feedback, which enables flexible feedback provision, does seem to hold promise for the future" (Yang & Carless, 2013, p.292) Additionally, studies indicate that students respond positively to technology-mediated feedback (Dodd and Kusaka, 2014) Furthermore, utilizing web-based learning management systems in interpreter training fosters improved feedback and interaction between teachers and learners, thereby enhancing collaborative learning experiences (Lee et al.).
A new approach to providing feedback in speaking classes involves the use of audio and note-taking technology, which has been shown to offer significant advantages for in-class feedback Applications like GoREACT and Audionote allow teachers to record their comments and take notes simultaneously, delivering time-coded and constructive feedback without interrupting students' fluency practice This innovative method addresses the ongoing challenges of decontextualized feedback commonly found in interpreting classes, making it a noteworthy development in educational practices.
To provide effective feedback, teachers should focus on its function, content, and presentation, while also considering student characteristics and contextual factors like learning goals and tasks (Shute, 2008) While timely and regular feedback is advantageous, the content of the feedback is crucial for its overall effectiveness (Bangert-Drowns et al., 1991; Scott).
Research indicates that effective feedback must be specific and goal-oriented, as highlighted by Voerman et al (2014) Detailed suggestions for improvement are far more beneficial than feedback that only addresses accuracy and adequacy (Bangert-Drowns et al 1991; Scott 2014) Shute (2008) supports this perspective, noting that vague feedback can yield minimal positive outcomes for learners Overall, a comprehensive review of the literature identifies key qualities of effective feedback: it should be specific, goal-related, timely, and regularly provided.
Previous research on formative feedback in English language teaching has largely overlooked the importance of addressing both academic and emotional needs of students This gap is particularly significant considering the stress students face during interpretation training Understanding this aspect has been crucial for the researcher in designing a study aimed at improving feedback quality in her interpreting course.
Interpreting
According to AIIC — the International Association of Conference
Interpreters facilitate communication by conveying spoken messages from a source language (SL) to a target language (TL), addressing the language barriers that exist between speakers of different languages (Interpreters, 2012) As highlighted by Jones (2002), interpreting is fundamentally about communication, and it also involves navigating diverse cultural backgrounds and knowledge bases that can complicate understanding The interpreter's role is to ensure that the meaning is accurately conveyed to the target audience, which may involve providing explanations or adapting references to align with the speaker's intended message (Jones, 2002, p 3).
In the interpreting process, communication occurs instantaneously, requiring interaction among speakers, listeners, and interpreters This dynamic practice utilizes specific linguistic resources, where the original speaker's ideas are conveyed through spoken words characterized by distinct rhythm and intonation, alongside the use of rhetorical devices and gestures.
Simultaneous interpreting involves real-time translation as the speaker talks, while consecutive interpreting occurs after the speaker pauses, allowing interpreters to convey the message.
Simultaneous interpreting can be defined as the procedure of "cross- linguistic transfer of meaning in real-time (Seeber, 2011) The term
The term "simultaneous" pertains not to the concurrent understanding and articulation of a specific sentence component, but rather to the overall temporal overlap between language comprehension and production.
Simultaneous interpreting is a highly demanding task for interpreters, as it requires the simultaneous processing of language comprehension and production According to Seeber (2011), this real-time integration leads to competition for cognitive resources, resulting in increased stress for interpreters.
2.2.2 Rubrics for assessing interpreting performances
This paper explores the quality of feedback in interpreter training by investigating various criteria for assessing interpreting quality, aiming to establish a framework for providing effective feedback to students A comprehensive marking rubric for simultaneous interpreting is essential for evaluating pre-test and post-test performances, addressing the first research question Consequently, this literature review will analyze different theories to develop both a detailed marking rubric and a simplified feedback rubric specifically for Vietnamese-English simultaneous interpreting.
2.2.2.1 Assessment criteria for interpreting performances
Despite the increasing research on interpreting quality, there remains a lack of consensus on the criteria for assessing it While some studies have addressed performance quality assessment in interpreting, few have explored the fundamental definition of quality itself.
Firstly, in his paper on the International Association of Conference message‖, ―Logical cohesion of utterance‖, ―Correct grammatical usage‖,
―Completeness of interpretation‖, ―Fluency of delivery‖, ―Native accent‖, and
Kahane (2000) identified two distinct groups—interpreters and users—who evaluated the significance of various criteria for assessing interpreting quality Both groups ranked the criteria similarly, emphasizing that sense consistency with the original message and logical cohesion are the most critical factors In contrast, a native accent and pleasant voice projection were deemed the least important aspects of interpreting quality.
In their 2004 article, "Interpreters’ Perception of Linguistic and Non-Linguistic Factors Affecting Quality: A Survey through the World Wide Web," Chiaro and Nocella presented a quantitative study examining professional interpreters' views on interpreting quality Utilizing Buler's (1986) framework for assessment criteria, they categorized these criteria into linguistic factors—such as consistency with the original speech, completeness of information, and fluency of delivery—and extra-linguistic factors, including concentration, teamwork ability, and physical well-being Through an online survey targeting interpreters across five continents, the study revealed that consistency with the original speech, completeness of information, and logical cohesion were deemed the most critical linguistic criteria, while concentration emerged as the top non-linguistic factor.
Zwischenberger (n.d.) conducted a study titled "Quality criteria in simultaneous interpreting: an international vs a national view," which explored conference interpreters' evaluations of various assessment criteria for simultaneous interpreting The research aimed to understand interpreters' perceptions of how the significance of these criteria correlates with changes in assignment types, comparing insights from the International Association of Conference Interpreters (AIIC) and the German Association of Conference Interpreters (VKD) Both groups rated sense consistency with the original and logical cohesion as the most important criteria Furthermore, the studies by Chiaro and Nocella (2004) and Zwischenberger (n.d.) share similarities and differences, as both seek to reveal interpreters' views on assessment criteria's importance, with a consensus that sense consistency is paramount, aligning with Bühler's (1986) findings.
The key distinction between the studies by Chiaro and Nocella (2004) and Zwischenberger (n.d.) is the framework employed to classify the criteria for evaluating the quality of interpretation.
In 2004, Chiaro and Nocella adapted Buler's (1986) assessment framework, categorizing criteria into linguistic (e.g., consistency with the original, completeness of information, fluency) and extra-linguistic (e.g., concentration, teamwork, physical well-being) aspects, creating a clear and accessible structure In contrast, Zwischenberger's study lacks a cited source for her framework and focuses solely on linguistic criteria, organizing them into three groups: content-related, form-related, and delivery-related criteria Notably, while many specific criteria overlap with Chiaro and Nocella's framework, Zwischenberger includes the ambiguous criterion of synchronicity.
2.2.2.2 Marking rubrics for interpreting performances
A marking rubric is essential for maintaining consistency in the quality assessment process, as it allows for reliable and uniform grading of students' performances in both pre-tests and post-tests.
A comprehensive review of the literature has revealed various studies focusing on criteria for quality assessment interpretation and their relative significance However, there is a notable lack of complete marking rubrics specifically designed for interpreting practices The following highlights key studies addressing this gap in research.
The Proposed Criteria for Peer-Feedback for Conference Interpreter Trainees, available on the Online Resources for Conference Interpreter Training (ORCIT) website, outlines a comprehensive rubric for evaluating interpreting performance This rubric features four primary macro-criteria, each framed as a question, and is further detailed by 6 to 13 specific micro-criteria For example, the second question, which focuses on the quality of the target language, includes 13 distinct sub-criteria.
Audionote and its application in giving feedback for speaking classes
The Audionote application represents a significant advancement in mobile learning by merging notepad and voice recorder functionalities This innovative tool enhances note-taking efficiency and quality, allowing teachers to record students' speaking performances and provide contextual feedback According to Dodd and Kusaka (2014), the app enables educators to send tailored speaking activities, thereby improving student engagement and understanding in language learning.
The researcher recognized the potential of this application to enhance feedback in interpreting performance, particularly in speaking activities that require students to translate messages from a source language to a target language Consequently, Audionote has been chosen as the primary tool to effectively address the challenges associated with providing feedback in the interpretation course.
Review of related studies
Numerous studies have explored the impact of computer-assisted feedback on students' spoken performance; however, there is a notable lack of research in this area within Vietnam, particularly concerning interpreting courses.
Research indicates that computer-assisted feedback offers numerous benefits, particularly in the field of interpreter training In their study "Effective Interpreter Feedback and Instruction Using GoREACT," Smith and Harris (2014) highlight how advanced technologies enhance the capabilities of interpreter trainers, facilitating more effective instruction and feedback.
Recent research highlights the advantages of utilizing advanced technology, such as GoREACT, to enhance the learning experience for tech-savvy students compared to traditional physical media recordings Smith and Harris (2014) emphasize the effectiveness of time-coded feedback and commentary, which facilitates improved communication between students and interpreter trainers, ultimately supporting students' development and growth.
Another action research study about ―Using Web 2.0: Synchronizing technology to improve feedback on spoken academic texts‖ by Dodd and Kusaka
In 2014, researchers at La Trobe Melbourne explored the use of Notepad and Voice Recorder technology, specifically Audionote, to enhance teacher feedback on students' speaking activities The study highlighted several advantages of this application, noting its capability to capture and synchronize spoken and written content effectively.
‗texts‘ into a digital file, deployable on computers and smart devices such as phones and tablets‖ According to Dodd and Kusaka (2014):
This innovative technology allows teachers to record students' speaking activities along with feedback notes, which can be sent to students for later review Students can replay these recordings and see highlighted feedback comments synchronized with their speaking, enabling them to understand the context of the feedback This approach caters to diverse learning styles and helps teachers ensure that their feedback is retained in its original context.
The integration of Notepad and voice recorder technology, such as Audionote, has demonstrated significant benefits in providing feedback during speaking classes However, there is a notable lack of research focused on the application of Audionote for delivering feedback to interpreter trainees This highlights a critical research gap concerning the effectiveness of Audionote and the responses of student interpreters to teacher feedback facilitated by this app.
RESEARCH METHODOLOGY
Research questions
This study sought to obtain data which helped to answer the following research questions:
Research question 1: What were the opinions of the students about the use of Notepad and Voice Recorder technology (Audionote) in the teacher’s feedback for the students’ classroom interpreting performances?
Research question 2: To what extent did the application of Notepad and Voice Recorder technology (Audionote) in the teacher’s feedback improve the students’ interpreting performances?
Research question 3: What were the difficulties of the teacher when applying Notepad and Voice Recorder technology (Audionote) in giving feedback for the students’ classroom interpreting performances (if any)?
Research approach
This thesis follows action research approach, with an in-depth analysis of the use of Audionote in the teacher's feedback for the students' interpreting practices in the classroom setting
Reasons for choosing action research approach
Action research is defined as a systematic study that combines action and reflection to enhance practice (Ebbutt, 1985) It involves a small-scale intervention in real-world scenarios, closely examining the effects of such interventions (Cohen, Manion & Morrison, 2007) Throughout the action research process, researchers engage in careful planning, acting, observing, and reflecting, applying a more rigorous approach than typical everyday life (Kemmis and McTaggart, 1992).
Zuber (1996) highlights that the primary goals of action research are to foster practical improvements, innovations, and changes in social practices, while also enhancing practitioners' understanding of their own methods In alignment with this, Kemmis and McTaggart support the idea that action research is essential for driving meaningful development and reflection in professional practices.
(1992) claim that action research is an approach to improve education by
"changing" it and learn from the results of changes
Action research aligns perfectly with the goals of this study, providing a solid rationale for its selection as the research approach This project aims to implement a new intervention—Notepad and Voice Recorder technology (Audionote)—to address the issue of decontextualized and ineffective teacher feedback in the classroom Through systematic reflection during the research process, I will gather data on students' perceptions of Audionote, ultimately aiming to enhance my feedback methods in interpreting courses In summary, action research is the ideal approach for this study.
Potential pitfalls of action research
While action research offers numerous benefits, such as connecting theoretical knowledge with practical application and fostering enhancements in teaching and learning, it is important to acknowledge the potential drawbacks highlighted by some scholars.
Subjectivity in action research can significantly impact its reliability, as noted by Cohen, Manion, and Morrison (2007) When the researcher is also the teacher, it becomes challenging to eliminate personal bias, which can influence the outcomes of the study.
A significant challenge in action research is the absence of a solid theoretical framework Since this approach primarily focuses on practical application, action researchers may find themselves without a strong theoretical basis to support their studies.
The limited generalizability of action research is an important factor to consider, as it typically takes place within a specific educational context and involves a small number of participants, often confined to a single class This restricts the ability to apply findings to different settings and broader populations.
Measures taken to overcome those potential pitfalls of action research
To handle the above-mentioned possible demerits of action research, I have implemented several measures
To enhance the reliability of the study and reduce subjectivity, both researcher triangulation and method triangulation were employed Researcher triangulation involved two independent raters, including a faculty colleague and myself, who evaluated students' simultaneous interpreting skills through pre- and post-intervention tests Additionally, a senior lecturer observed my class during the intervention phase, providing objective feedback through an interview Method triangulation was achieved by utilizing multiple data collection instruments, allowing for a more comprehensive understanding of the findings Specifically, student surveys and interviews were utilized to address Research Question 1 (RQ1), while pre- and post-intervention tests, along with surveys and interviews, were used for Research Question 2 (RQ2) Finally, Research Question 3 (RQ3) was informed by data from teaching journals and an interview with a classroom observer.
To address the lack of a theoretical framework, I conducted an extensive literature review on feedback, covering its definition, categorization, and characteristics of effective feedback, alongside interpretations of relevant studies on interpreting, including definitions and assessment rubrics for interpreting performances, as well as research related to Audionote.
To address the issue of generalizability, I meticulously selected participants and detailed my specific educational context The participants in my study represent typical students majoring in Translation and Interpreting at my university Consequently, the research findings are expected to be relevant for other students in the same field at my university or those in similar educational settings.
Data collection instruments
The study utilized multiple data collection instruments, including pre- and post-intervention tests, surveys, semi-structured interviews, classroom observer interviews, and teaching journals This diverse approach, integrating both quantitative (surveys) and qualitative methods (interviews and journals), facilitated triangulation of results, offering a comprehensive understanding of the situation The subsequent section will provide a detailed description of each data collection instrument employed in this research.
3.3.1 Pre-intervention tests and post-intervention tests
The pre-intervention and post-intervention tests conducted in this study were intricate and time-consuming, involving multiple steps to ensure a reliable evaluation of students' interpreting performance These tests serve as an effective means for comparing groups and measuring changes resulting from specific treatments or interventions (Dimitrov & Rumrill, 2003) This paper aims to assess the extent to which the use of Audionote contributed to improving interpreting skills.
At the start of the course, students participated in pre-intervention tests where they interpreted a selected speech for version 1 Their performances were graded and recorded using standard recording applications.
Students initially received traditional feedback from their teacher without the aid of Audionote Following this, they revised their performance at home and recorded a second version in class, which they then submitted for evaluation.
In the post-intervention tests, the students were asked to interpret another speech for version 1’ (the performance was graded and recorded using
The teacher utilized Audionote to provide both oral and written feedback to students, which was then sent via email for them to review at home This allowed students to listen to the feedback and make necessary revisions to their performance Subsequently, the students recorded a second version of their work during the post-intervention stage and submitted it to the teacher for evaluation.
A thorough analysis was conducted on the pre-intervention and post-intervention test results to assess the impact of feedback, with and without Audionote, on students' interpreting performance The assessment criteria were specifically adapted to identify any significant differences in improvement following the interventions.
3.3.2 Pre-intervention survey and post-intervention survey
The study employed pre and post-surveys to evaluate interpreting students' perceptions of teacher feedback in their classes, both prior to and following the intervention phase To ensure the validity of the results, the questionnaires were anonymous, allowing students to respond without providing their names The data gathered primarily addressed research questions one and two.
3.3.3 Pre-intervention interviews and post-intervention interviews
To gain deeper insights into students' evaluations of teacher feedback on their interpreting practices, semi-structured interviews were conducted before and after the intervention phase This qualitative approach was designed to complement the initial quantitative assessment effectively.
– survey so that the researcher clearly understood the students‘ responses and perceptions The instrument was also used to find the answer to RQ 1 and RQ 2
3.3.4 While-intervention classroom observer interview
A classroom observer interview was conducted to gather insights from a senior interpreter trainer regarding the intervention process This expert was invited to observe the class and subsequently participated in a semi-structured interview to evaluate the effectiveness of the intervention During the interview, she discussed challenges faced and proposed solutions for enhancing the intervention's outcomes This approach primarily addressed Research Question 3 and added a collaborative element to the study, enhancing its objectivity and reliability (Wallace, 2001, p 41).
Teaching journals served as the primary tool for documenting the teaching process and facilitating systematic reflections during the research According to Burns (2010), journals are vital for capturing significant reflections and events continuously This classic instrument in action research enables teachers to record specific events while also reflecting on their beliefs and insights regarding their practices.
89) Similarly, Wallace (2001) asserts that journals are an ―excellent tool for reflections‖ that ―promotes the development of reflective teaching‖
In her analysis of journal types, Burns (2010, p.90) emphasizes the value of descriptive and reflective journals These journals serve to document not only factual events but also personal thoughts, reflections, and insights They provide crucial input for researchers, enabling them to modify and enhance teaching methods throughout the research process, ultimately leading to improved educational outcomes.
Following the guidelines and questions outlined by Burns (2010, p.90), I customized the inquiries and developed a template for the teaching journal utilized in this study, as demonstrated in the excerpt from my teaching journal presented in the table below.
Table 3: An extract of the teaching journal
What are my perceptions/ attitudes about the happening?
What are my responses/ interpretations of what has happened?
The students were asked to deliver version 2 of the pre-test in class, record their performance and submit it to the teacher
+ I conducted preliminary questionnaires and interviews The survey was about the students‘ evaluation of previous teachers‘ feedback for their interpretation in the past
+ I introduced the students to Audionote I
+ The students seemed to be interested in the new app
+ The app is quite easy to use for them
+ There was serious lack of time for pre- interviews with the students
+ The survey results were quite satisfactory but there was still room for development The students needed more feed-forward feedback
To address time constraints during pre-interviews, I opted to conduct individual phone interviews with students outside of class This approach effectively allowed me to engage with my students and gain a deeper understanding of their needs The convenience of phone interviews benefited both my students and myself, as I encouraged them to install the necessary app on their devices.
(preferably their personal computer) me because it was time- saving yet still efficient
I used the app for in- class feedback without any feedback criteria or rubric:
+ called student (S)1-3 to do simultaneous interpreting (3-5 minutes for each S);
+ recorded the interpreting performances and made comments, using
+ showed the recording with notes for the whole class for detailed feedback after each S had finished,
Ineffective feedback often stems from time constraints and the absence of a clear feedback rubric I struggled to maintain pace with the students during simultaneous interpreting, as I had to concentrate on both the source and target speeches simultaneously This made it challenging to compare the two and take notes for feedback while they were speaking.
+ Usually, I could only point out the students‘ mistakes in small details but lacked general
In response to the emerging need for a feedback rubric, I developed a plan for Cycle 2 of the study, utilizing the Audionote app for feedback Throughout the week, I thoroughly examined the literature on quality assessment, noting the diverse perspectives on this topic My division has yet to establish a consensus on a feedback rubric for students, prompting me to adopt insights from various studies to create a user-friendly rubric for teachers This rubric is designed to streamline the commenting process on student interpretations, allowing me to focus more on listening to their speeches and providing constructive feedback Ultimately, I recognized the necessity for a feedback rubric tailored for student interpreters, featuring specific criteria and a dedicated section for general comments.
I continued to use the app together with the new feedback rubric
The experience was better but not as expected
My feedback procedure was still time-consuming if I replayed the whole recording of each student‘s performance in class
Data collection procedure
This action research was conducted over a 12-week period, specifically from week 2 to week 13 of the selected semester, which typically spans 15 weeks The research was planned for after the first week to ensure that both students and the teacher were familiar with one another and had grasped essential course information.
Action research is characterized by a cyclical model proposed by Kemmis and McTaggart (1988), which includes four key phases: planning, action, observation, and reflection This model allows for a continuous spiral of cycles, enabling researchers to refine their approach until they achieve satisfactory results The recursive nature of this process is visually represented in Figure 1, showcasing the iterative steps involved in action research.
Figure 1 Cyclical action research model
Despite facing criticism for being overly rigid and prescriptive, the aforementioned model remains a valuable tool that effectively summarizes the key stages of an action research study (Burns, 2010, p.8).
Nunan's seven-step model of action research, outlined in 1992, includes initiation, preliminary investigation, hypothesis, intervention, evaluation, dissemination, and follow-up This comprehensive approach aligns well with the researcher's objectives, making it a suitable choice for the study Furthermore, the research will adhere to the recursive and cyclical nature of action research as proposed by Kemmis and McTaggart, indicating that multiple cycles may be necessary to achieve the desired outcomes.
The procedure consisted of two cycles as follow:
I observed and realized the inadequacies in giving feedback for the interpreting class (from week 1 – 3)
I surveyed the students for their assessment of teachers‘ feedback in their interpretation classes Subsequently, I conducted follow-up interviews for further information about the students‘ assessment of teachers‘ feedback
The survey questionnaire utilized in this study was adapted from the research conducted by Dodd and Kusaka (2014), titled "Using Web 2.0: Synchronizing Technology to Improve Feedback on Spoken Academic Texts." For more details regarding the preliminary questionnaire, please refer to Appendix 4, page XI.
The hypotheses of the research study are:
Hypothesis 1: Using audio and note-taking technology (Audionote) in teacher’s feedback may improve the quality of the teacher’s feedback
Hypothesis 2: Using audio and note-taking technology (Audionote) in teacher’s feedback may help the students to enhance their interpreting skills
Hypothesis 3: The teacher may encounter some certain difficulties in applying Audionote in giving feedback for interpreting students
Table 4: Intervention phase in Cycle 1 students install the app both on their smartphones and their laptops teaching journal during the process
Phase 2 I used the app for in-class feedback:
+ called 3 Ss to interpret for each session (3-5 minutes for each student);
+ recorded the interpreting and make comments, using Audionote;
+ after each student has finished, I showed the recording with notes for the whole class for detailed feedback;
+ sent the feedback to students for their own revision
I observed my own class and kept reflection in the teaching journal
After the first five steps, I reflected on the first cycle then planed for the next cycle
I faced a significant time constraint while providing traditional feedback to students after their initial pre-intervention test To ensure that all students received feedback before submitting their second version the following week, I opted to deliver comments in class for some students and communicate with others through phone calls immediately after the lesson This approach proved to be the most effective way to manage my time and ensure that every student received the necessary feedback.
During the initial weeks of using Audionote, I found that my feedback notes were ineffective due to time constraints and the absence of a specific feedback rubric for quality assessment To address this issue, I decided to create a tailored feedback rubric for my teaching context, which I planned to implement in the second cycle of my research procedure.
First, I looked through the literature and developed a feedback rubric for
An interview was conducted with a senior interpreter, who is also a key lecturer in the field of interpretation, to explore the advantages and disadvantages of using Vietnamese-English interpreting in the classroom This discussion aims to provide insights into the effectiveness of this approach in enhancing students' language skills and understanding of cultural nuances.
After refinement, the rubric was copied and pasted in Audionote files for me to use in the classrooms to keeps my notes on
Table 5: Intervention phase in Cycle 2
Phases Description of the phase Notes
Phase 1 I continued to use the app for in-class feedback with the newly-developed feedback rubric:
+ called 3 students to interpret for each session (3-5 minutes for each Ss);
+ recorded the interpreting and make comments, using Audionote;
+ after each student had finished, showed the recording with notes for the whole class for detailed
+ The teacher- researcher kept a teacher’s journal during the process
- Invited one revision where the researcher uses Audionote in giving feedback;
- Interviewed the senior interpreter trainer for comments on the use of Audionote
Phase 2 Students‘ revision & reflection based on the teacher‘s recorded feedback:
I asked students to choose one of the recordings and record their revision, compare, and reflect whether their performance has been improved
Phase 3 I surveyed the students about the effectiveness of using Audionote for improving teacher‘s feedback
(See Appendix 5 for further information about the post-questionnaire)
Follow-up interviews were conducted with students about the effectiveness of using Audionote for improving teacher‘s feedback
(See Appendix 6 for further information about the interview questions)
Phase 4 I involved one other colleague in the
The Translation and Interpreting Division will evaluate students' pre-test and post-test results to analyze their progress A colleague has been tasked with assessing the recordings of 20 students using a specific set of criteria tailored by the researcher.
(Refer back to page 20 for the full version of the rubric)
Each student‘s recording was marked twice: first by myself and second by another teacher in the same division This would assure the objectivity of the results
During the observation phase, a senior interpreter trainer was invited to observe an interpreting class where the researcher utilized Audionote for feedback Following the observation, interviews were conducted with the senior interpreter trainers to gather their insights on the effectiveness of using Audionote in the feedback process.
In the end, I reflected on the two cycles, then drew conclusions.
Data analysis procedure
Quantitative analysis was utilized for both pre-intervention and post-intervention tests and surveys, while qualitative methods were employed for other data collection tools, including interviews and teaching diaries.
3.5.1 Pre-intervention tests and post-intervention tests
In the initial phase of data collection, both pre-intervention and post-intervention tests were evaluated by two independent raters, including a faculty colleague and myself, who used a consistent marking system for the recordings.
After the scoring process, the marks from two raters were compared If the difference between their scores for a student's version was between 0 and 1 point (out of 10), both scores were accepted, and the final score was the average of the two However, if the disparity exceeded 1 point, the raters discussed the case and reassessed the performance The final scores were then recorded in Microsoft Excel for further analysis and calculation.
Paired sample t-tests were also done in SPSS software to calculate the mean, standard deviation (SD), etc and most importantly the p-value (sig.)
The Paired Sample t-test is utilized to compare the means of two related groups, such as professionals at different career stages, learners taking pre-tests and post-tests, or subjects assessing body mass index before and after an exercise program (Ross & Willson, 2017) A statistically significant difference between the means is indicated when the p-value (sig.) is less than the level of significance (α = 0.05).
The analysis of the pre-intervention test scores indicates that if the p-value exceeds the significance level of 0.05, there is no statistically significant difference between the means of the two variables.
(the pre-intervention test scores and pre-intervention test scores)
A more detailed explanation about the Paired sample t-test can be found in Appendix 3 on page 96
3.5.2 Pre-intervention survey and post-intervention survey
After collecting survey papers from the students, the researcher manually typed the survey responses for close-ended questions into Survey Monkey to receive a visual illustration of the results
To analyze open-ended questions, the researcher employed sticky notes and a coding technique to categorize responses into themes and subthemes This process included the use of marginal notes and color coding, ensuring that all responses within the same theme or sub-theme shared a consistent color code Examples illustrating the coding process are provided in the accompanying pictures.
Figure 2: Coding examples for open-ended questions in the surveys
3.5.3 Interviews (Pre-intervention and post-intervention interviews with the students and while-intervention interview with the classroom observer)
In analyzing qualitative data from interviews, the researcher utilized end-use strategizing to align objectives with the creation of a narrative that effectively summarizes the overall evaluation and interpretation of the data.
To effectively communicate the intended narrative to readers, a coding scheme was developed that utilized a hybrid approach, combining both pre-set and emergent codes The pre-set codes were derived from the research questions, conceptual framework, and identified problem areas, while emergent codes surfaced through the analysis of interview transcripts Key pre-set codes for the interview analysis included "influences on teacher feedback," "influences on interpretation skills," "frequency of review and reflection," "importance of having a marking rubric," "shortcomings," and "expectations/suggestions."
―evaluation of previous traditional feedback‖
Colour coding was also used mostly with emergent codes The pieces of information which belonged to the same code would have the same colour
The process of analyzing data from the interviews can be summarized in the following flow chart:
Figure 3: Process of analyzing data from the interviews
Following is an example of the coding process:
Figure 4: Examples of the coding process
Listenning to the interviews for the first time then creating a storyline
Summarizing and organizing the interview scripts using the pre-set codes
Adding emergent codes, color codes, and marginal remarks while refining the codes
The same analyzing procedure with coding, colour coding and note-taking happened with data from the teaching journal.