INTRODUCTION
Statement of research problem & questions
In recent decades, there has been a growing interest in the impacts of language assessment, commonly referred to as test washback (Alderson & Wall, 1993; Bailey, 1996; Hughes, 1989; Caine, 2005) Research on this topic spans various countries, including Canada, Brazil, Germany, Greece, Japan, China, and Hong Kong, with numerous studies focusing on prominent tests such as TOEFL, IELTS (Alderson & Hamp-Lyons, 1996; Green, 2007), the Hong Kong Certificate of Education Exam in English (HKCEE) (Cheng, 2004), and the General English Proficiency Test (Pan & Newsfields, 2012) Findings regarding the degree of washback have shown significant variation over time; while some researchers reported minimal washback (Watanabe, 2014; Shih, 2007), others observed a strong influence (Ferman, 2004) Therefore, it appears that washback effects can differ significantly across various contexts.
In recent years, the importance of foreign languages in Vietnam has significantly increased due to the National Foreign Language 2020 project, which aims for most students graduating from secondary schools, vocational schools, colleges, and universities to confidently use a foreign language in daily communication by 2020 As a result, educational institutions are required to implement a Graduation Proficiency Assessment (GPA) for their students One prominent assessment is the Vietnam Standardized Test of English Proficiency (VSTEP), a high-stakes test that impacts various stakeholders, including students and educators However, there is a lack of research on the washback effects of GPA and VSTEP, particularly concerning students who are not majoring in English.
This study aims to explore the impact of washback on the preparation strategies and attitudes of non-English major students at VNUH's School of Law regarding the graduation proficiency assessment (GPA).
In brief, this study will address the following questions:
1 What are students’ perceptions toward the English graduation proficiency assessment?
2 To what extent does the graduation proficiency assessment influence students’ preparation strategies(from students’ perspective)?
3 Are there differences in the test preparation strategies of students in different years?
Scope of research
This research focuses on students at the School of Law, Vietnam National University, Hanoi (VNUH), who must achieve a GPA level of B1 or B2 to graduate This requirement translates to passing level 3 or level 4 of the Vietnamese Standardized Test of English Proficiency (VSTEP) Alternatively, students may submit accepted certificates such as TOEFL, TOEIC, Cambridge Test, or IELTS Ultimately, students have the flexibility to take any examination that meets the equivalent requirements of a level 3 or 4 VSTEP certificate to fulfill their language graduation criteria.
For time limited, this study will only look at the washback on students’ preparation for the test and their attitude toward the GPA.
Significance
This study, while limited in scale, aims to address the gap in literature regarding washback effects on non-English major students By increasing awareness of various test preparation strategies, it seeks to enhance their learning experience Additionally, it provides valuable insights for teachers, practitioners, and stakeholders to better understand student attitudes and learning processes, ultimately improving the quality of tests and educational activities Furthermore, this research serves as a resource for future studies in language assessment, specifically in the area of washback.
Design of the study
This study is divided into five chapters:
Chapter 1: Introduction – presents the rationale, aims research questions, significance, scope, and design of the study
Chapter 2: Literature review – discusses the theoretical framework and studies of washback and English language proficiency tests
Chapter 3: Methodology – demonstrates the context and research design of the study
Chapter 4: Findings and discussion – shows the comprehensive of data and a discussion on the findings of the study
Chapter 5: Conclusion – provides a summary of the findings, conclusion, recommendation, limitations, and future direction for the study
LITERATURE REVIEW
Over the past 30 years, language testing has undergone significant evolution, particularly in the 1980s when it transitioned from discrete-point formats to communicative language assessments This period also marked an expansion into Second Language Acquisition The following decade continued this growth, focusing on various aspects such as research methodology, practical advancements, performance-influencing factors, and authenticity in testing.
The ethical issues surrounding test use, particularly the consequences of testing, remain a significant topic of discussion globally Among these issues, the concept of washback—how tests influence teaching and learning—has garnered considerable attention Researchers argue that the impact of testing can significantly shape individuals' educational futures, highlighting the critical role assessments play in educational systems.
In their 2010 study, Brown and Abeywickrama highlight the significant influence of standardized tests on educational policy and behavior promotion, as noted by Shohamy et al in 1996 The authority of these assessments often leads policymakers to utilize them as tools for managing the educational system.
2.2 Washback in language testing and assessment
Testing has long been integral to education and employment, serving various purposes such as assessing proficiency, acting as gatekeepers, and motivating students Given its significant role, testing is believed to influence both learning and teaching Prior to the 1990s, key concepts in language testing included "curricular alignment," which emphasizes the connection between testing content and the curriculum; "measurement-driven instruction," which posits that testing should guide curriculum and teaching; and "systemic validity," which focuses on adapting curriculum and instruction to enhance the cognitive skills that tests aim to measure.
1989, p.27) In 1993, for the first time, Alderson and Wall official introduced the
5 term “washback” with proven evidence from empirical studies Since then, more attention has been paid to washback with different definitions
Washback or backwash in language assessment is generally known as
The concept of "washback" refers to the influence of testing on teaching and learning, as noted by Hughes (1989) Although this term is significant in the field of applied linguistics, it is not commonly found in dictionaries Some, like the New Webster’s Comprehensive Dictionary and the Collins Cobuild Dictionary, include a related term, "backwash," which is described as the negative consequences of certain social actions.
The term "unpleasant after-effect of an event or situation" highlights a negative implication associated with the adverse relationship between testing and teaching/learning, as discussed by Cheng et al.
2004) Alderson & Wall (1993) assume that washback renders “teachers and learners to do things they would not necessarily otherwise do” (p.117) Messick
Washback, as defined by Shohamy et al (1996), refers to the impact that tests have on language learning and teaching practices, influencing educators and learners to engage in activities that they might not otherwise undertake This concept highlights the significant relationship between testing and learning, emphasizing how assessments can either promote or hinder the language acquisition process.
Washback can be categorized into narrow and broad scopes, with the narrow scope primarily focusing on the impact of testing on teaching and learning According to Alderson and Wall (1993), Bailey (1996), and Messick (1996), tests influence both teachers and learners to engage in activities they might not otherwise pursue, highlighting the significant role assessments play in educational practices.
Washback is a crucial aspect of consequential validity, which refers to all the consequences of a test, including its accurate measurement criteria, influence on test preparation, and the social implications of its interpretation and use (Brown & Abeywickrama, 2010) Bachman and Palmer (1996) describe this concept as "impact," which encompasses the various consequences of assessment both before and after test administration Consequently, washback is viewed as a specific dimension of this broader impact (Hamp-Lyons, 1997).
Testing impacts can be analyzed at two distinct levels: micro and macro At the micro level, the effects are felt by individuals, such as test-takers and teachers Conversely, the macro level encompasses the broader implications of testing, influencing the entire educational system and society as a whole.
This study focuses on the narrow interpretation of "washback," specifically examining its micro-level effects on individuals It investigates how the VSTEP test influences students, particularly in terms of their test preparation strategies and attitudes towards the assessment.
The nature of washback from a test is often categorized as either positive or negative; however, Watanabe (2000) identifies four additional dimensions: specificity, which can be general or specific; intensity, described as strong or weak; length, which refers to being long or short; and intentionality.
(intended or unintended) (Cheng at al., 2004, p.20) In this minor thesis, only the value and specificity of washback are discussed
Washback, as defined by Alderson & Wall (1993), is a neutral term that can yield both positive and negative effects on education Positive washback enhances teaching and learning activities, fosters positive attitudes towards testing, and boosts student motivation Shohamy (1993) supports this view, noting that tests can encourage students to engage more deeply in their studies, such as listening attentively Conversely, negative washback manifests as test anxiety and an overwhelming focus on tested skills, leading to fear of poor performance (Tsagari, 2011) This pressure may cause teachers to narrow their curriculum and concentrate solely on test preparation Additionally, students may resort to reviewing only test-related materials, as highlighted by Damankesh & Babaii (2015) Ultimately, positive washback promotes beneficial educational changes, while negative washback results in detrimental effects.
Washback can be categorized as either general or specific General washback occurs when tests motivate students to study harder overall, while specific washback is evident in a focused area or type of test According to Didi & Ridha (2011), the English National Examination (ENE) exemplifies specific washback, as teachers prioritize reading skills—an essential component of the test—over communication skills.
This part examines different models of washback of Hughes (1993), Bailey
(1996), Aldersom & Wall (1993) and Shih (2007) Firstly, a basic model of washback been early proposed by Hughes (1993) in which three components
“participants”, “process” and “products” are differentiated (Bailey, 1996, p.262)
Hughes (1993) emphasizes that a variety of participants, including students, classroom teachers, administrators, materials developers, and publishers, can have their perceptions and attitudes towards their work influenced by assessments.
METHODOLOGY
In Vietnam, the importance of English has significantly increased due to Project 2020 initiated by the Prime Minister in 2008 This initiative aims to ensure that most Vietnamese graduates from universities and colleges can effectively use English in daily conversations To support this goal, the VSTEP (Vietnamese Standardized Test of English Proficiency) with five levels was introduced, requiring university students to pass specific levels of the test for graduation Additionally, internationally recognized certificates such as IELTS, TOEFL, and TOEIC are also accepted, further promoting English proficiency among students.
The School of Law at VNUH requires students to pass the VSTEP level 3 or an equivalent international test as a graduation proficiency assessment (GPA), with fast-track students needing to reach level 4 Certificates from these tests are valid for two years, prompting most students to schedule their assessments during the latter half of their studies Additionally, students have the flexibility to take the test whenever it suits them best.
This study examines 3 research questions:
1 What are students’ perceptions toward the English graduation proficiency assessment?
The research explores how students perceive tests and their impact on their attitudes, revealing whether these perceptions lead to positive or negative influences Additionally, students' views on their GPA may shed light on their behaviors and motivations.
2 To what extent does the graduation proficiency assessment influence students’ preparation strategies (from students’ perspective)?
This study explores students' reflections on their behaviors during the preparation phase, aiming to determine the influence of GPA on their actions It assesses whether this impact is strong or weak, positive or negative, and whether it is general or specific.
3 Are there differences in the test preparation strategies of students in different years?
The question is raised to explore whether the first-year students and senior study English differently for the GPA
3.3 Participants and selection of participants
The population size is approximately 1400 students including around 700 first-year and second year students and roughly 700 third-year and fourth-year students at the School of Law, VNUH
Students at the School of Law, VNUH, are not majoring in English or studying English for Special Purposes The institution offers three General English courses ranging from level A1 to B1 for mainstream students, along with an additional B2 course for fast-track students Enrollment in these courses is optional throughout their university journey, but graduation proficiency assessments are mandatory Mainstream students must achieve a B1 level, while fast-track students need to reach B2 by graduation Students' proficiency levels are determined through VSTEP certificates or other recognized international tests such as TOEIC, IELTS, and TOEFL, which they must complete before graduating First-year and second-year students have ample time to prepare for these assessments.
In comparison, the third-year and four-year students are those who have limited time for test preparation
The study employs a stratified sampling design, dividing the population into two groups based on school year: first and second-year students in one stratum, and third and fourth-year students in another, maintaining an approximate 1:1 ratio Cluster sampling is utilized within each stratum, with law students attending classes in Hall P1 and P2 of G Building, allowing for random selection of specific classes for data collection.
The sample size was 123 students considering the population size 1400 students, confidence level 95%, confidence interval ±8
In a survey of 123 students, 82.9% (102 students) reported taking or having taken the GPA at school, while 17.1% expressed interest in alternative proficiency tests such as IELTS and TOEIC Students who opted not to take the GPA will be excluded from the second half of the study, which focuses on test preparation behavior Among the 102 students who participated in the GPA, 57 were first or second-year students, representing 55.9%, while 45 were third or fourth-year students, accounting for 44.1%.
In summary, the sample is reasonably representative of the School of Law GPA test-taking students population
Eight respondents from the sample will be invited for the interview after finishing the questionnaires They are those who are willing to take part in the interview round
This study utilizes face-to-face questionnaires and semi-structured interviews due to their specific advantages The observation method is not used, as there is no mandatory English course at the School of Law, allowing students the flexibility to choose classes that best suit their needs.
The study addresses four research questions using a questionnaire as the primary instrument due to its significant advantages Questionnaires are particularly effective for reaching a large number of respondents, making them ideal for the sample of 123 participants, as they facilitate the distribution of surveys, analysis of responses, and formulation of general conclusions more efficiently than one-on-one interviews, group discussions, or observational methods Additionally, this approach is cost-effective, allowing researchers to swiftly gather a substantial amount of data from participants.
The face-to-face questionnaire is preferred for its administrative benefits, allowing researchers to engage directly with participants to clarify the study's purpose and address any immediate questions This method also ensures a higher response rate compared to online and mail questionnaires.
The questionnaire, adapted from Stoneman (2005), includes questions about students' self-assessment and attitudes towards the test, as well as their out-of-class activities and preparation methods It is structured into four distinct sections.
Part 1 asks for demographic information Questions about the name, phone, and email are not numbered and students only need to leave information if they are willing to join the interview round Question 1’s purpose is to find out whether students take the GPA or not All the questions done by non-GPA test- takers will be excluded As the purpose of this paper is to find the washback of GPA, if students take other tests, it is unlikely that they have any influences Question 2 collects students’ school year which helps the researcher to divide the participants into two groups as presented above in order to answer research question 3
Part 2 helps to collect washback on attitudinal aspects There are 7 questions from question 3 to 9 Question 3 asks for the reasons why students choose GPA but not other tests The next two questions are students’ self-rating of their English proficiency and test anxiety which might explain students test preparation later The next four questions seek for students’ perspectives of GPA as a source of motivation, as a means to raise students’ standard, as a tool to measure students English proficiency correctly and a support for job application
Part 3 includes 2 questions to find out students’ learning activities outside classes; to be more specific whether students learn English by watching the TV program, listening to music, or speaking with native speakers The second question aims at examining the average time students spend on doing those activities weekly
Part 4: includes 4 questions which are to investigate washback on students’ behavioral and attitudinal aspects
- demographic information (name, phone, and email)
- reasons to take the GPA (believes toward the test)
- respondents’ level of test anxiety
Perspective toward the test as:
- as source of motivation for students to improve their English
- as a means to help raise students’ English standard
- as a tool to measure students’ English proficiency
- the usefulness of the test result in supporting job applications
- time spent on those activities per week
- time spent preparing for the test
- types of test preparation activities
As participants cannot be directly observed, interview is a reasonable method to collect participants’ historical information (Creswell, 2014) such as the process that students prepare for the test
Semi-structured interview is also utilized because there are disadvantages in the use of questionnaires Firstly, it is argued to be insufficient in
To enhance understanding of behaviors and feelings, the questionnaire employs a fixed scheme with limited questions, potentially omitting vital information To address these limitations, a semi-structured interview approach is utilized, allowing for more comprehensive data collection.
RESULTS AND DISCUSSION
This section outlines the findings regarding students' attitudes toward their GPA and test anxiety, derived from questionnaire responses It also incorporates insights from interview data According to the methodology, the first research question, "What are students' attitudes toward the graduation proficiency assessment?" is addressed through six specific questionnaire items.
- Why do you take the VSTEP instead of IELTS, TOEFL, TOEIC or other compatible tests as the graduation test? (Q3)
- Do you think the graduation proficiency assessment (GPA) will motivate students to improve their English? (Q6)
- Do you think the GPA will raise the proficiency of English of university students? (Q7)
- Do you think GPA will measure students’ English proficiency accurately? (Q8)
- Do you think the GPA test result will be useful in helping you find a job? (Q9)
- How do you feel about having to take the GPA? (Q5)
The answers for these questions are presented below
4.1.1 Students’ decision to take the GPA
Respondents stated their reasons to choose GPA by answering an open- ended question This type of question helps the scholar to collect numerous
27 opinions, but its disadvantage is the low response rate; in return, there were only
57 answers which are categorized into 7 groups as in figure 4.1 below
A significant portion of respondents (31.2%) viewed the GPA as the simplest proficiency test when compared to TOFEL, IELTS, and TOEIC Additionally, the GPA was considered convenient for registration and affordable for non-English majors at the School of Law.
Figure 4.1 Reasons to take the GPA
A significant portion of students, specifically 31.2%, reported taking the test due to school requirements Interviews indicated that many students were unaware of alternative tests available for transferring their scores.
For personal reasons, one student stated that she trusted the test designers in the University of Languages and International Studies (ULIS)
In conclusion, the GPA is perceived as a more suitable and less challenging option for students compared to alternative proficiency tests Many respondents indicated that they opted for the GPA because it is a requirement set by their school, making it the easiest choice available.
4.1.2 Whether the test would motivate students to improve their English
Respondents were asked to answer yes/ neutral/ no question to the question “Do you think the graduation proficiency assessment (GPA) will
Reasons to take the GPA
A recent study revealed that 56.3% of Law School students believe that a high GPA serves as a motivating factor for improving their English skills, while only 22.7% disagreed, and 21% remained neutral This indicates a positive trend among students, suggesting that the pursuit of a better GPA encourages them to enhance their English proficiency Follow-up interviews provided further insights into this perspective.
Figure 4.2 Students’ perception toward test motivation
Students believed that passing the GPAS was essential for earning their degrees and graduating from university, as well as fulfilling the requirements for English courses, which motivated them to study diligently for the exam.
“Like other tests, the GPA forces me to study however boring English is to me”
“if it had not been for the GPA, I would not have studied English” (GPA_4)
One participant indicated that the GPA stimulated her to study the language further
“The test informs me about my English proficiency, so I know where I am, what are my strengths and weaknesses Therefore, I know what’s more I should learn.”GPA_2
Regarding the claim that the test does not provoke learning, some participants found their wants, the test’s level, and test importance as the culprit
“I didn’t want to learn English, instead I wanted to spend time learning Chinese The test just stressed me out but not inspire me” GPA_4
“I find the GPA not too challenging; besides, I am confident with my English level, so I don’t spend much time to learn it” GPA_3
In short, most respondents found the test motivate them to improve their English; whereas, some students considered the test unimportant and unchallenge to stimulate learning
4.1.3 Whether the test would raise students’ English proficiency
A survey conducted on university students regarding the impact of the VSTEP test on English proficiency revealed a largely positive outlook According to the results, 63.1% of respondents believe that VSTEP will enhance their English skills, while only 17.5% disagreed This indicates a strong enthusiasm among students about the potential benefits of the test in improving their language proficiency.
Figure 4.3 Students’ perception toward test function of raising English proficiency
Many interviewees expressed disagreement with the findings, with respondents across varying levels of English proficiency—both low and high—claiming that their English skills did not improve as a result of the test.
“I find the test easy so I merely spend time for it I think my English proficiency does not raise at all.” GPA_3
Achieving a B1 level in English has been challenging for me, so I enrolled in an English center where experienced teachers assist me with the test format While I receive templates for speaking and writing, I feel that this approach lacks practical application, as it does not enhance my ability to communicate effectively Consequently, I believe that my experience at the center has not improved my overall competence in the language.
Even though many students insisted on gaining no more knowledge from learning for the test, throughout the interview, the researcher could spot their minor improvements
During the speaking test, I comprehended most of the phrases and concepts expressed by the teachers; however, I struggled with fluency in my responses After leaving the room, a multitude of ideas surged through my mind, leaving me with a sense of regret for not sharing my thoughts.
Upon entering university, this student realized she had little knowledge of English However, her ability to speak even a bit of English, albeit imperfectly, marked a significant improvement in her language skills.
The questionnaire results indicate that many respondents view the test as a valuable tool for enhancing students' English proficiency; however, interview participants expressed a more pessimistic perspective This discrepancy may stem from the interviewees' lack of perceived progress in their language learning journey.
4.1.4 Whether the test would measure students’ proficiency accurately
Respondents provided their opinions on the accuracy of VSTEP in measuring students' English proficiency by answering yes, neutral, or no to the question posed It is important to note that these individuals are not experts in testing or language, which limits the validity of their responses as evidence of the test's accuracy The primary aim of the question was to gauge students' perceptions regarding the test's reliability and face validity To ensure clarity, a more accessible term was used in place of technical jargon, allowing students to better articulate their views on the quality of the test.
The chart illustrates students' varying opinions on test accuracy, revealing that nearly half of the respondents either did not choose an option or left their answers blank Among those who did express a view, the responses were nearly evenly split, with 30.1% affirming accuracy and 27% indicating otherwise.
Figure 4.4 Students’ perception of the GPA’s accuracy
The interview data conveyed an insight explanation for these answers Many students did not rate the test accurate as they were not familiar with the test
“I am not clear about the GPA format and structure so I cannot give you an answer”
Some thought that this question should be answered by teachers and expert instead of students As they found themselves not entitled and trustworthy to deem the test
“I don’t know, I am not an expert.”
For students who held positive perspective, the test structure and content seemed to persuade them
“Well I think the GPA is fairly accurate as it assesses four skills listening, speaking, reading and writing like many other proficiency tests.”
The GPA test is more suitable for Vietnamese individuals compared to other proficiency tests like IELTS, TOEFL, and TOEIC, which can be challenging due to their foreign influences In contrast, the GPA provides a context that is more relatable and familiar to local students.
However, regarding the reliability of individual skills, many students expressed personal doubts toward MCQs and assessing speaking skills
Perceive the test as an accuracy tool to measure students' proficiency
“I think reading, listening, and writing tests are quite okay, but I am not sure about speaking skills, the scores vary following the examiner.” GPA_2
“I don’t think multiple choice questions can assess students’ levels accurately, as they can choose randomly.”
Others believed that the overall score did not reflect the test-taker English proficiency