INTRODUCTION
Statement of research problem & questions
The impact of language assessment, or test washback, has garnered significant attention in recent decades across various research contexts, including Canada, Brazil, Germany, Greece, Japan, China, and Hong Kong Numerous studies have focused on large-scale tests such as TOEFL, IELTS, and the Hong Kong Certificate of Education Exam in English, revealing varying degrees of washback effects While some researchers found minimal washback in their studies, others identified a strong influence, suggesting that the effects of washback can differ significantly depending on the specific context.
In recent years, the importance of foreign languages in Vietnam has significantly increased due to the National Foreign Language 2020 project, which aims for most Vietnamese students graduating from secondary schools, vocational schools, colleges, and universities to confidently use a foreign language in daily communication and professional settings by 2020 Consequently, universities and colleges are required to implement a Graduation Proficiency Assessment (GPA) for their students, with the Vietnam Standardized Test of English Proficiency (VSTEP) being a key high-stakes assessment that impacts students, teachers, and practitioners However, there is a lack of research on the washback effects of GPA or VSTEP on students, especially those not majoring in English.
This study aims to explore the impact of washback on the preparation strategies of non-English major students for the graduation proficiency assessment (GPA) at VNUH, specifically within the School of Law, as well as their attitudes toward the test.
In brief, this study will address the following questions:
1 What are students’ perceptions toward the English graduation proficiency assessment?
2 To what extent does the graduation proficiency assessment influence students’ preparation strategies(from students’ perspective)?
3 Are there differences in the test preparation strategies of students in different years?
Scope of research
This research focuses on students at the School of Law, Vietnam National University, Hanoi (VNUH), who must achieve a GPA of B1 or B2 to graduate This requirement translates to passing level 3 or level 4 of the Vietnamese Standardized Test of English Proficiency (VSTEP) Alternatively, students can submit accepted certifications such as TOEFL, TOEIC, Cambridge Test, or IELTS, as long as they demonstrate an equivalent proficiency level to the VSTEP requirements for graduation.
For time limited, this study will only look at the washback on students’ preparation for the test and their attitude toward the GPA.
Significance
This study, while limited in scale, aims to address the literature gap regarding washback effects on non-English major students By exploring various test preparation strategies, it seeks to enhance students' awareness and improve their learning outcomes Additionally, the findings will provide teachers and stakeholders with valuable insights into student attitudes and learning processes, ultimately contributing to the enhancement of test quality and educational practices Furthermore, this research will serve as a resource for future studies in language assessment, particularly in the area of washback.
Design of the study
This study is divided into five chapters:
Chapter 1: Introduction – presents the rationale, aims research questions, significance, scope, and design of the study
Chapter 2: Literature review – discusses the theoretical framework and studies of washback and English language proficiency tests
Chapter 3: Methodology – demonstrates the context and research design of the study
Chapter 4: Findings and discussion – shows the comprehensive of data and a discussion on the findings of the study
Chapter 5: Conclusion – provides a summary of the findings, conclusion, recommendation, limitations, and future direction for the study
LITERATURE REVIEW
Over the past 30 years, language testing has significantly evolved, transitioning from discrete-point formats in the 1980s to a focus on communicative language testing This period also saw an expansion into Second Language Acquisition The following decade continued this trend, exploring various areas such as research methodology, practical advancements, and factors influencing performance on language tests, while emphasizing the importance of authenticity.
The ethical issues surrounding test use, particularly the consequences of testing, remain a significant topic of discussion globally, as highlighted by Bachman (200, p.4) Among these concerns, the concept of washback—how tests influence educational practices—has garnered attention for its critical role in shaping educational outcomes Researchers like Shohamy emphasize that the impact of testing can significantly govern and determine individuals' future educational paths.
In 2001, Brown and Abeywickrama highlighted the significant influence of standardized tests, which are often leveraged by policymakers to shape educational systems and encourage specific behaviors (Shohamy et al., 1996).
2.2 Washback in language testing and assessment
Testing has played a significant role in education and employment for various purposes, including assessing proficiency, serving as gatekeepers, and motivating students This influence of testing on learning and teaching has led to concepts such as "curricular alignment," which connects testing content with the curriculum; "measurement-driven instruction," which suggests that testing should guide curriculum and teaching; and "systemic validity," which emphasizes the need for curriculum and instructional changes to develop the cognitive skills that tests aim to measure.
1989, p.27) In 1993, for the first time, Alderson and Wall official introduced the
5 term “washback” with proven evidence from empirical studies Since then, more attention has been paid to washback with different definitions
Washback or backwash in language assessment is generally known as
The concept of washback, referring to the impact of testing on teaching and learning, is significant in the field of applied linguistics (Hughes, 1989) Although the term is not commonly found in dictionaries, some, like the New Webster’s Comprehensive Dictionary and the Collins Cobuild Dictionary, include a related term, "backwash," which denotes the negative consequences of social actions.
The term "unpleasant after-effect of an event or situation" highlights the negative implications associated with the relationship between testing and teaching/learning, as discussed by Cheng et al.
2004) Alderson & Wall (1993) assume that washback renders “teachers and learners to do things they would not necessarily otherwise do” (p.117) Messick
Washback, as defined by Shohamy et al (1996), refers to the influence that testing has on language instruction and learning, impacting teachers and learners in ways that may either promote or hinder language acquisition (p.241) This concept underscores the crucial relationship between assessment and educational outcomes.
Washback is categorized into narrow and broad scopes, with the narrow scope focusing on the impact of tests on teaching and learning According to Alderson and Wall (1993), Bailey (1996), and Messick (1996), tests influence both teachers and learners, prompting them to engage in activities they might not otherwise pursue.
Washback is a key aspect of consequential validity, which addresses the various outcomes of a test, including its effectiveness in measuring intended criteria, its impact on test preparation, and the social implications of test interpretation and use (Brown & Abeywickrama, 2010) Bachman and Palmer (1996) refer to this broader concept as impact, which encompasses the consequences of assessment both before and after test administration Consequently, washback is recognized as a specific dimension of impact (Hamp-Lyons, 1997).
Testing impacts can be analyzed on two levels: micro and macro At the micro level, the effects are felt by individuals such as test-takers and teachers Conversely, the macro level reflects the broader implications of testing on the entire educational system and society as a whole.
This study focuses on the narrow interpretation of "washback," specifically examining its micro-level effects on individuals It will analyze how the VSTEP test influences students, particularly in terms of their test preparation strategies and attitudes towards the exam.
The washback effect of a test is often categorized as either positive or negative, but Watanabe (2000) identifies four additional dimensions: specificity, which refers to whether the impact is general or specific; intensity, indicating the strength of the effect; length, which describes the duration of the influence; and intentionality, reflecting the purpose behind the washback.
(intended or unintended) (Cheng at al., 2004, p.20) In this minor thesis, only the value and specificity of washback are discussed
Washback, as defined by Alderson & Wall (1993), is a neutral concept that can yield both positive and negative effects Positive washback enhances teaching and learning activities, fosters favorable attitudes towards assessments, and boosts student motivation In contrast, negative washback can lead to test anxiety and an excessive focus on tested skills, as noted by Tsagari (2011) This anxiety may cause teachers to concentrate solely on test results, potentially narrowing the curriculum Additionally, students might resort to reviewing only materials directly related to the test, as highlighted by Damankesh & Babaii (2015) Ultimately, while positive washback promotes beneficial changes, negative washback results in undesirable outcomes.
Washback can be categorized as either general or specific, with general washback affecting students' overall study habits, leading them to study harder for any test In contrast, specific washback is evident in particular aspects of a test According to Didi & Ridha (2011), the English National Examination (ENE) exemplifies specific washback, as teachers emphasize reading skills— a major component of the test— over communication skills.
This part examines different models of washback of Hughes (1993), Bailey
(1996), Aldersom & Wall (1993) and Shih (2007) Firstly, a basic model of washback been early proposed by Hughes (1993) in which three components
“participants”, “process” and “products” are differentiated (Bailey, 1996, p.262)
Hughes (1993) emphasizes that a variety of stakeholders, including students, classroom teachers, administrators, materials developers, and publishers, can have their perceptions and attitudes influenced by testing This highlights the significant impact that assessments can have on the educational environment and the individuals within it.
METHODOLOGY
In Vietnam, the importance of English has significantly increased due to Project 2020 initiated by the Prime Minister in 2008 This initiative aims to ensure that most Vietnamese graduates from universities and colleges are proficient in English for daily conversations To support this goal, the VSTEP (Vietnamese Standardized Test of English Proficiency) with five levels has been established, and university students are required to pass specific levels of this test to graduate Additionally, internationally recognized certificates such as IELTS, TOEFL, and TOEIC are also accepted as proof of English proficiency.
The School of Law at VNUH requires students to pass the VSTEP level 3 or an equivalent international test as a graduation proficiency assessment (GPA), while fast-track students must achieve level 4 Certificates from these tests are valid for two years, leading most students to opt for the exam during the latter half of their studies, with the flexibility to take the test whenever it suits them.
This study examines 3 research questions:
1 What are students’ perceptions toward the English graduation proficiency assessment?
This research explores students' perceptions of tests and their influence on academic performance, revealing whether these perceptions are positive or negative Additionally, understanding students' views on their GPA may shed light on their behaviors and attitudes towards their studies.
2 To what extent does the graduation proficiency assessment influence students’ preparation strategies (from students’ perspective)?
This study explores students' reflections on their behaviors during the preparation phase, examining the extent to which GPA influences their actions It aims to determine whether the effects of GPA are strong or weak, and whether they are perceived as positive or negative, as well as general or specific.
3 Are there differences in the test preparation strategies of students in different years?
The question is raised to explore whether the first-year students and senior study English differently for the GPA
3.3 Participants and selection of participants
The population size is approximately 1400 students including around 700 first-year and second year students and roughly 700 third-year and fourth-year students at the School of Law, VNUH
Students at the School of Law, VNUH, are not majoring in English or studying English for Special Purposes The school offers three General English courses ranging from A1 to B1 for mainstream students, along with an additional B2 course for fast-track students Enrollment in these courses is optional, but all students must undergo a compulsory graduation proficiency assessment Mainstream students are required to achieve a B1 level, while fast-track students must reach B2 by graduation The school evaluates students' English proficiency through VSTEP certificates or other recognized international tests such as TOEIC, IELTS, and TOEFL, which students must complete before graduating First and second-year students, being new to the school, have ample time to prepare for these assessments.
In comparison, the third-year and four-year students are those who have limited time for test preparation
The study employs a stratified sampling design, dividing the population into two strata based on school year: first and second-year students in one group, and third and fourth-year students in another, with a roughly equal ratio of 1:1 Cluster sampling is utilized for each stratum, focusing on law students who attend different classes in Hall P1 and P2 of G Building, from which specific classes will be randomly selected for data collection.
The sample size was 123 students considering the population size 1400 students, confidence level 95%, confidence interval ±8
In a survey of 123 students, 82.9% (102 students) reported taking or having taken the GPA at school, while 17.1% expressed interest in alternative proficiency tests such as IELTS and TOEIC Those not participating in the GPA will be excluded from the latter part of the study, which focuses on students' test preparation behaviors Among the 102 students who took the GPA, 57 were first and second-year students, comprising 55.9%, while 45 were third and fourth-year students, accounting for 44.1%.
In summary, the sample is reasonably representative of the School of Law GPA test-taking students population
Eight respondents from the sample will be invited for the interview after finishing the questionnaires They are those who are willing to take part in the interview round
This study utilizes face-to-face questionnaires and semi-structured interviews due to their distinct advantages The observation method is not employed, as the School of Law does not require a compulsory English course, allowing students the flexibility to choose classes that best fit their needs.
The study employs a questionnaire as its primary research instrument, effectively addressing four key research questions This method is particularly advantageous for reaching a large sample size of 123 participants, making it more manageable than one-on-one interviews or group observations Additionally, questionnaires are cost-effective, allowing researchers to swiftly gather a significant amount of data from respondents, facilitating comprehensive analysis and conclusions.
The face-to-face questionnaire is preferred for its administrative advantages, allowing researchers to personally engage with participants, clarify the study's purpose, and address any immediate questions This method also ensures a higher response rate compared to online and mail questionnaires.
The adapted questionnaire from Stoneman (2005) includes inquiries about students' self-assessments and attitudes towards the test, as well as their extracurricular activities and preparation methods It is structured into four distinct sections.
Part 1 asks for demographic information Questions about the name, phone, and email are not numbered and students only need to leave information if they are willing to join the interview round Question 1’s purpose is to find out whether students take the GPA or not All the questions done by non-GPA test- takers will be excluded As the purpose of this paper is to find the washback of GPA, if students take other tests, it is unlikely that they have any influences Question 2 collects students’ school year which helps the researcher to divide the participants into two groups as presented above in order to answer research question 3
Part 2 helps to collect washback on attitudinal aspects There are 7 questions from question 3 to 9 Question 3 asks for the reasons why students choose GPA but not other tests The next two questions are students’ self-rating of their English proficiency and test anxiety which might explain students test preparation later The next four questions seek for students’ perspectives of GPA as a source of motivation, as a means to raise students’ standard, as a tool to measure students English proficiency correctly and a support for job application
Part 3 includes 2 questions to find out students’ learning activities outside classes; to be more specific whether students learn English by watching the TV program, listening to music, or speaking with native speakers The second question aims at examining the average time students spend on doing those activities weekly
Part 4: includes 4 questions which are to investigate washback on students’ behavioral and attitudinal aspects
- demographic information (name, phone, and email)
- reasons to take the GPA (believes toward the test)
- respondents’ level of test anxiety
Perspective toward the test as:
- as source of motivation for students to improve their English
- as a means to help raise students’ English standard
- as a tool to measure students’ English proficiency
- the usefulness of the test result in supporting job applications
- time spent on those activities per week
- time spent preparing for the test
- types of test preparation activities
As participants cannot be directly observed, interview is a reasonable method to collect participants’ historical information (Creswell, 2014) such as the process that students prepare for the test
Semi-structured interview is also utilized because there are disadvantages in the use of questionnaires Firstly, it is argued to be insufficient in
To enhance the understanding of behaviors and feelings, the questionnaire's fixed scheme may lead to gaps in important information To address these limitations, a semi-structured interview approach is employed.
RESULTS AND DISCUSSION
This section outlines the findings regarding students' perspectives on GPA and test anxiety, derived from the questionnaire responses It also includes additional insights from interview data In alignment with the methodology chapter, the first research question—“What are students’ attitudes toward the graduation proficiency assessment?”—is addressed through six specific questionnaire items.
- Why do you take the VSTEP instead of IELTS, TOEFL, TOEIC or other compatible tests as the graduation test? (Q3)
- Do you think the graduation proficiency assessment (GPA) will motivate students to improve their English? (Q6)
- Do you think the GPA will raise the proficiency of English of university students? (Q7)
- Do you think GPA will measure students’ English proficiency accurately? (Q8)
- Do you think the GPA test result will be useful in helping you find a job? (Q9)
- How do you feel about having to take the GPA? (Q5)
The answers for these questions are presented below
4.1.1 Students’ decision to take the GPA
Respondents stated their reasons to choose GPA by answering an open- ended question This type of question helps the scholar to collect numerous
27 opinions, but its disadvantage is the low response rate; in return, there were only
57 answers which are categorized into 7 groups as in figure 4.1 below
A significant portion of respondents (31.2%) considered the GPA to be the most accessible proficiency test compared to TOFEL, IELTS, and TOEIC Additionally, the GPA was viewed as a convenient option for registration and an affordable choice for non-English majors at the School of Law.
Figure 4.1 Reasons to take the GPA
A significant 31.2% of respondents indicated that they took the test due to school requirements Additionally, interviews revealed that many students were unaware of alternative tests available for transferring their scores.
For personal reasons, one student stated that she trusted the test designers in the University of Languages and International Studies (ULIS)
In conclusion, the GPA appears to be a more suitable option for students compared to alternative proficiency tests, as it is often seen as less challenging Many respondents indicated that they take the GPA test because it is a requirement set by their school, and it is regarded as the easiest choice available.
4.1.2 Whether the test would motivate students to improve their English
Respondents were asked to answer yes/ neutral/ no question to the question “Do you think the graduation proficiency assessment (GPA) will
Reasons to take the GPA
A recent study revealed that 56.3% of Law School students believe that a high GPA serves as a motivating factor for improving their English skills, while only 22.7% disagreed Additionally, 21% of respondents were neutral on the matter These findings suggest a positive correlation between GPA and students' motivation to enhance their English proficiency Further insights from interviews support this perspective.
Figure 4.2 Students’ perception toward test motivation
Students believed that passing the GPAS was essential for earning their degrees and graduating from university, as well as for fulfilling requirements in English courses, which motivated them to dedicate time to their studies.
“Like other tests, the GPA forces me to study however boring English is to me”
“if it had not been for the GPA, I would not have studied English” (GPA_4)
One participant indicated that the GPA stimulated her to study the language further
“The test informs me about my English proficiency, so I know where I am, what are my strengths and weaknesses Therefore, I know what’s more I should learn.”GPA_2
Regarding the claim that the test does not provoke learning, some participants found their wants, the test’s level, and test importance as the culprit
“I didn’t want to learn English, instead I wanted to spend time learning Chinese The test just stressed me out but not inspire me” GPA_4
“I find the GPA not too challenging; besides, I am confident with my English level, so I don’t spend much time to learn it” GPA_3
In short, most respondents found the test motivate them to improve their English; whereas, some students considered the test unimportant and unchallenge to stimulate learning
4.1.3 Whether the test would raise students’ English proficiency
A survey was conducted to gauge students' opinions on whether the VSTEP test would enhance university students' English proficiency The results reveal a generally positive outlook, with 63.1% of respondents expressing agreement, while only 17.5% disagreed.
Figure 4.3 Students’ perception toward test function of raising English proficiency
Many interviewees expressed disagreement with the findings, with respondents across varying levels of English proficiency—both low and high—reporting no improvement in their English skills as a result of the test.
“I find the test easy so I merely spend time for it I think my English proficiency does not raise at all.” GPA_3
Achieving a B1 level in English is challenging for me, so I attend an English center where teachers experienced with the test format offer assistance Instead of focusing on fundamental skills like grammar and vocabulary, I receive templates for speaking and writing However, these templates are impractical for real communication, leading me to believe that the program does not enhance my overall competence.
Even though many students insisted on gaining no more knowledge from learning for the test, throughout the interview, the researcher could spot their minor improvements
During the speaking test, I comprehended the majority of the phrases and concepts presented by the teachers; however, I struggled with fluency in my responses Once I exited the room, a multitude of thoughts flooded my mind, leaving me regretting that I hadn't expressed my ideas more effectively.
Upon entering university, this student realized she had little knowledge of English However, even her ability to speak a few words, albeit not fluently, marked a significant improvement in her language skills.
The questionnaire results indicate that many respondents view the test as a tool for enhancing students' English proficiency; however, interview participants express a more pessimistic outlook This discrepancy may stem from the interviewees' inability to recognize their own progress in language learning.
4.1.4 Whether the test would measure students’ proficiency accurately
Respondents indicated their perceptions of VSTEP's accuracy in measuring students' English proficiency by answering yes, neutral, or no to the question posed It's important to recognize that these individuals are not language or testing experts, which means their responses cannot serve as definitive evidence of the test's accuracy Instead, the aim of the question is to understand students' perceptions of the test's quality The term "accuracy" encompasses aspects like face validity and reliability; however, since students may not be familiar with technical jargon, a more accessible term was used.
The chart illustrates students' varied responses regarding test accuracy, revealing that nearly half of the participants either chose not to respond or left their answers blank Among the responses, those affirming test accuracy accounted for 30.1%, while those disagreeing represented 27%, indicating a near-equal distribution of opinions among students.
Figure 4.4 Students’ perception of the GPA’s accuracy
The interview data conveyed an insight explanation for these answers Many students did not rate the test accurate as they were not familiar with the test
“I am not clear about the GPA format and structure so I cannot give you an answer”
Some thought that this question should be answered by teachers and expert instead of students As they found themselves not entitled and trustworthy to deem the test
“I don’t know, I am not an expert.”
For students who held positive perspective, the test structure and content seemed to persuade them
“Well I think the GPA is fairly accurate as it assesses four skills listening, speaking, reading and writing like many other proficiency tests.”
The test is better suited for Vietnamese individuals compared to other proficiency tests like IELTS, TOEFL, and TOEIC, which are shaped by foreign perspectives and can be challenging In contrast, the GPA provides a more relatable context for assessing language proficiency.
However, regarding the reliability of individual skills, many students expressed personal doubts toward MCQs and assessing speaking skills
Perceive the test as an accuracy tool to measure students' proficiency
“I think reading, listening, and writing tests are quite okay, but I am not sure about speaking skills, the scores vary following the examiner.” GPA_2
“I don’t think multiple choice questions can assess students’ levels accurately, as they can choose randomly.”
Others believed that the overall score did not reflect the test-taker English proficiency