Introduction 1.1 Rationale
Scope of study
This thesis focuses on evaluating the content validity of the final achievement test by comparing its contents with the objectives, syllabus, and textbook allocation The study analyzes data from the currently utilized test and offers practical recommendations for its improvement.
Due to constraints in time, resources, and capabilities, the writer is unable to address all aspects of the tests; therefore, only select recommendations for test improvements are provided.
Aims of study
This study evaluates the content validity of the final achievement test designed for second-year non-major students at Phuong Dong University, focusing on a thorough analysis of the test's content.
The specific aims of this research are:
- To find out the strengths and weaknesses of the currently used test with reference to the content validity
- To suggest some improvements for the test.
Methods of study
In order to achieve the above-mentioned aims, a combination of many methodologies was utilized
The author grounded her analysis in language testing theories and the essential characteristics of effective assessments, emphasizing content validity Through extensive review of reference materials, she established a theoretical framework to evaluate the achievement test administered to second-year students, specifically focusing on its alignment with the curriculum from their first semester This evaluation aims to assess the test's content validity in relation to the students' prior learning.
Qualitative methodologies utilizing survey questionnaires were implemented to gather insights from English teachers and students at Phuong Dong University Two distinct sets of questionnaires were distributed to assess their evaluations regarding the content validity of the final achievement test, along with suggestions for potential improvements.
Research questions
In this study, the writer tries to answer the two following questions:
Question 1: What are the strengths and weaknesses of the final achievement test with reference to the content validity for second year non major students at Phuong Dong University?
Question 2: What are some suggested solutions for the improvements of the test?
Design of study
The thesis is organized into five major chapters:
1 Chapter 1 INTRODUCTION presents such basic information as: the rationale, the aims, the methods, the research questions and the design of the study
2 Chapter 2 LITERATURE REVIEW presents a review of related literature that provides the theoretical basis for evaluating and building a good language test This review includes background on language testing, criteria of good tests and theoretical issues on test content validity
3 Chapter 3 THE STUDY mentions the methods used in the research and which shows its detailed results of the surveys including the questionnaires and the analysis of the final achievement test in order to find out its problems with reference to the content validity
4 Chapter 4 RECOMMENDATIONS AND CONCLUSIONS Recommendations provide some suggestions for the improvements of the final achievement test basing on the mentioned theoretical and practical study Conclusions summarize the matters of research, its findings as well as its limitations.
Literature review
The roles of language testing
Language testing is a form of measurement It helps the teachers:
Evaluating a learner's achievement in a language program involves assessing their language knowledge in relation to the specific curriculum or materials covered during the course.
Evaluating a learner's language proficiency is essential for determining their suitability for future language use, such as assessing whether an individual possesses the necessary language skills to work as a tourist guide This evaluation focuses on the practical application of the language, independent of the specific programs or materials the learner has previously engaged with.
+ To diagnose a learner‟s strengths and weaknesses in a language and to attempt to explain why the certain problems occur and what treatments could be used to tackle these problems
+ To classify or place the testees in the appropriate language classes
+ To measure the testee‟s aptitude for learning a language
To assess the effectiveness of a language program, researchers typically implement experimental and control classes that share the same educational goals but utilize distinct methods and materials to achieve these objectives (Brown, 2000:5).
Bebecca M Valette (1977) highlights three essential roles of classroom tests in second language teaching: they help define course objectives, stimulate student progress, and evaluate overall class achievement.
Classroom tests play a crucial role in defining course objectives, as students tend to adapt their study habits based on the types of assessments they encounter For instance, while a teacher may prioritize oral fluency during lessons, the prevalence of written tests will lead students to focus more on enhancing their reading and writing skills.
Tests play a crucial role in enhancing student progress by providing rewarding experiences that allow students to demonstrate their proficiency in the target language Unlike in the past, tests should not focus on highlighting students' weaknesses, but rather be announced in advance to ensure adequate preparation It is essential for students to receive prompt feedback on their performance after the test, as this reinforces learning by confirming correct answers and addressing errors.
Testing serves a crucial role in assessing student achievement by allowing teachers to identify challenges faced by individual students and the class overall By analyzing test errors, educators can focus on targeted drills and tailor support for each student Additionally, testing provides insight into whether class objectives are being met and evaluates the effectiveness of teaching methods, approaches to challenging concepts, and new materials Ultimately, classroom tests offer an objective measure of each student's progress, reflecting their attainment of course objectives and performance compared to peers.
Relationship between testing and teaching- learning
Historically, teaching and testing have been viewed as separate entities, both theoretically and practically Williams (1983) argues that testing is an unpleasant necessity outside the classroom for two primary reasons Firstly, testing fosters competition rather than cooperation; while classroom activities often promote collaboration through group work, tests require individual performance, which can obscure the true abilities of each participant Secondly, tests create a dichotomy of winners and losers, leading to discomfort for those who struggle, while those on the verge of success may feel less affected This competitive nature undermines the cooperative spirit that is typically encouraged in educational settings.
In recent years, there has been a significant shift towards integrative and communicative testing methods in English language assessment, leading to innovative approaches Researchers widely agree that teaching and testing are inherently interconnected, highlighting the importance of aligning assessment practices with educational objectives.
Teaching and testing are closely interconnected, making it challenging to separate them (1994) Tests serve as tools to reinforce learning, motivate students, and assess their performance in the language Essentially, a test extends classroom activities, offering valuable insights that enhance both teaching and learning Additionally, the teaching and learning processes generate rich language materials that can be effectively utilized in testing.
A well-designed test serves as an essential educational tool by providing teachers with insights into the effectiveness of their instruction It enables educators to assess students' abilities and understand individual characteristics Additionally, tests allow for ongoing monitoring and evaluation of student learning, helping to identify strengths and weaknesses in real-time Finally, the results from these assessments inform teachers about the efficacy of the syllabus, teaching methods, and materials utilized in the classroom.
Testing can significantly impact teaching and learning, both positively and negatively When a test is deemed crucial, the preparation for it often overshadows other educational activities Consequently, teachers may concentrate solely on test-related content, leading to a distortion of the overall teaching program.
2.3 Major characteristics of a good test
Before creating a test, it is essential to consider the key characteristics that define a good assessment According to Harrison (1983: 10), four fundamental traits are crucial: validity, reliability, practicality, and discrimination.
Validity is a crucial characteristic of an effective test and has been a topic of debate for many years Recently, discussions in language testing have shifted towards viewing validity as a unitary concept that encompasses various types, recognizing it as a multifaceted aspect of assessment.
Henning (1987:5) defines validity as follows:
Validity refers to how well a test measures what it claims to measure A test is considered valid if it accurately assesses the intended construct It's important to note that a test may be valid for specific purposes but not necessarily for others.
A test is deemed valid when it accurately measures what it is intended to assess For instance, a listening test with challenging written multiple-choice options may compromise its validity by also measuring reading comprehension, particularly for students who excel in listening but struggle with reading Validity is defined as the extent to which a test fulfills its intended purpose; for example, a test designed to evaluate English communication skills is valid only if it effectively measures that ability Additionally, validity exists on a spectrum, with some tests being more valid than others, and it is essential to consider the intended use of the test A test meant to assess reading ability may become invalid if it inadvertently evaluates writing skills as well.
Validity pertains to the accuracy and relevance of the conclusions drawn about individuals and groups based on test results It is essential to evaluate validity in relation to the precision of specific inferences regarding test takers, making it a complex aspect to assess effectively.
Validity is a crucial concept in research, encompassing various types, including face validity and content validity This article will concentrate specifically on these two main types, highlighting their significance in ensuring the accuracy and relevance of measurement tools.
Face validity refers to the extent to which a test appears, from the learners' perspective, to measure what it is intended to measure It is primarily assessed based on content; if a test accurately reflects what learners have achieved or are expected to achieve, it is considered to have face validity According to Arthur Hughes (1989), a test has face validity if it seems to assess the intended skills For instance, a test claiming to evaluate pronunciation but not requiring spoken responses would likely be viewed as lacking face validity Tests without face validity may face rejection from candidates, educators, and educational authorities Additionally, face validity is influenced by the perceptions of non-experts, including candidates, their families, and the public, and can be assessed by soliciting opinions from other teachers.
The rise of communicative language testing has led to a greater focus on face validity, emphasizing the importance of tests resembling real-world language use This connection to "real life" is crucial for establishing face validity While students' opinions may not be expert assessments, they provide valuable insights as feedback from test-takers If a test lacks perceived validity, candidates may not fully engage, making their perceptions an important factor in the testing process.
In other words, the face validity affects the response validity of the test This critical view of face validity provides a useful method for language test validation
Face validity serves as a quick and effective guide, addressing concerns related to statistical analysis while maintaining student motivation when a test is perceived as relevant If students find a test lacking in relevance, it will consequently lack face validity A test can encompass all components of a teaching program yet still fail to exhibit face validity Although the concept of face validity has been present in language testing for some time, its current emphasis is relatively new Previously, test writers viewed face validity merely as a public relations tool; however, contemporary communicative test designers now consider it a crucial aspect of overall test validity.
Among several kinds of validity, the simplest and most important one to the language teachers is content validity
According to Read (1983:6), content validity is the most crucial aspect of classroom testing, as it ensures that the test's content aligns with the syllabus objectives To accurately assess students' progress, tests must include a representative sample of the skills, rules, and items that students are expected to master Therefore, focusing on the test contents is essential for achieving content validity.
Kerlinger (1973) defines content validity is the representative or sampling adequacy of the content, the substance, the matter and the topics of a measuring instrument
In the same way, Harrison (1983: 11) defines content validity as:
Major Characteristics of a good test 1 Test validity
3.1 English learning, teaching and testing at Phuong Dong University 3.1.1 The students
At Phuong Dong University, students hail from various regions across the country, many of whom had limited exposure to English during high school due to their focus on subjects like mathematics, physics, and chemistry for university entrance exams As a result, they enter university as true beginners in English, with varying levels of language proficiency.
English teachers instructing second-year students vary in age, with half between 45 and 55 years old and the other half ranging from 25 to 38 years old These educators have graduated from three notable institutions: Ha Noi National University, Ha Noi Foreign Language University, and Phuong Dong University.
3.1.3 The course book: “New Headway Elementary- The third edition”
The textbook "New Headway Elementary - Third Edition" is utilized for teaching second-year students at Phuong Dong University, specifically tailored for those at the elementary level.
It consists of 14 units, designed in a harmonious combination with powerful lexical to increase learners‟ vocabulary and develop awareness of the English culture
Each unit consists of three sections that concentrate on grammar, function, and vocabulary Students can enhance their understanding of grammar, vocabulary, communication skills, and pronunciation through various practice activities, including listening, speaking, reading, and writing.
The study 3.1 English learning, teaching and testing at Phuong Dong University 3.1.1 The students
The teachers
English teachers instructing 2nd-year students come from diverse age groups, with 50% aged between 45 and 55, while the other half ranges from 25 to 38 years old These educators have graduated from three prominent institutions: Ha Noi National University, Ha Noi Foreign Language University, and Phuong Dong University.
The course book “New Headway Elementary- The third edition”
The "New Headway-Elementary- Third Edition" textbook is utilized for teaching second-year students at Phuong Dong University, specifically tailored for those at the elementary level.
It consists of 14 units, designed in a harmonious combination with powerful lexical to increase learners‟ vocabulary and develop awareness of the English culture
Each unit is structured into three sections that emphasize grammar, function, and vocabulary Students engage in various practice activities—listening, speaking, reading, and writing—to enhance their understanding of grammar, vocabulary, communication skills, and pronunciation.
Syllabus and its objectives
In the first semester of their second year, students engage in seven units from unit 7 to 14 over 45 periods, each lasting 50 minutes, within a span of approximately 9 weeks The curriculum focuses on enhancing grammar, vocabulary, communication skills, and pronunciation, while allowing students to explore various topics The primary objectives of the course are to bolster students' foundational knowledge of vocabulary and grammar, as well as to provide practice in the four essential language skills: listening, speaking, reading, and writing in social contexts.
The final achievement test
The final achievement test consists of the following parts: types, items, tasks
Part Types Items Tasks Marks
Part 1 Rewrite the sentences 5 Rewrite sentences so that there is no change of meanings
5 Use the following sets of words to write complete sentences
Part 3 Correct mistakes 5 Find and correct one mistake in each sentence
Part 4 Write a paragraph 1 Write a paragraph of 100-120 words about your capital city
Table 3: The components of the final achievement test
The marking criteria for the test have caused confusion among teachers and anxiety among students Specifically, part 4 poses a challenge for educators due to the absence of detailed evaluation guidelines, including aspects like language use, content quality, and grammar.
Research method
This study employs both quantitative and qualitative methods, primarily focusing on document analysis to assess the strengths and weaknesses of the final achievement test in terms of content validity Additionally, survey questionnaires are utilized to gather insights from both teachers and students regarding the test While each method contributes valuable data, it is important to acknowledge their inherent limitations.
Survey questionnaires are an effective method for data collection, particularly in assessing teachers' and students' attitudes towards the content validity of end-of-term tests They facilitate a non-confrontational environment, allowing informants to freely express their opinions through a structured list of questions Additionally, the prevalence of closed-ended questions simplifies data collection and analysis, making it easier to interpret results This method also enables the gathering of a large volume of responses, enhancing the reliability of the findings.
Besides survey questionnaires, document analysis is considered as the main method to evaluate the final achievement test in terms of the content validity
This article examines "The New Headway - Elementary - Third Edition" to determine the teaching requirements for educators and the learning expectations for students The study focuses on assessing the content validity of the final achievement test for second-year students at Phuong Dong University, utilizing test analysis as a key method By applying theories of testing, test design principles, and the characteristics of effective assessments, the writer will compare the course objectives and the material covered by students against the test content This analysis aims to identify the test's strengths and weaknesses, ultimately providing recommendations for enhancements.
Last but not least, the writer will analyze the data of survey questionnaires from both teachers and students to see how their comments about this test are
To ensure the authenticity of research findings, it is crucial to employ multiple methodologies for comparison This approach allows for the genuine expression of informants' feelings and perspectives Additionally, document analysis serves as a valuable resource, providing insights into the actual practices of teachers and students By combining document analysis with survey questionnaires, researchers can achieve objective and reliable results.
Data analysis
This section analyzes the final achievement test by comparing its content with the curriculum covered in the first semester, aiming to identify the strengths and weaknesses of the current test in terms of content validity Furthermore, the opinions of both students and teachers, gathered through survey questionnaires, are examined to provide a more comprehensive evaluation of the test's content validity.
3.3.1 Analysis of the final achievement test
The final achievement test for second-year non-major students consists of four key components that require careful examination of both its layout and content.
The final achievement test - No1
Time allowed: 60 minutes Total score Marker’s signature1:
I Rewrite each sentence, beginning as shown, so that the meaning stays the same
1 My watch is cheaper than yours
2 Mick is a very careful driver
3 What are your favorite things at the weekend?
4 No one is more intelligent than Anna in her class
5 Do you want some fish and chips?
II Guided sentence building: use the following sets of words and phrases to write complete sentences
1 I‟d/ chicken/and chips/main course
2 Everest/highest/mountain/the world?
3 You/been/Machu Pichu/before?
4 Alice/tall/long black hair/blue eyes
5 My sister/have/new car/but/drive/it
III Find and correct ONE mistake in each of the following sentences
1 My brother can play badminton when he was five years old
2 Who‟s is this jacket on your chair?
3 This morning, I had breakfast quick because I was late for school
4 It is good to see you Where have you gone?
5 Martin is very good at badminton He won the game easy
IV Writing topic: Write a paragraph of 100-120 words about your capital city
The final achievement test is a formal assessment designed to evaluate the overall learning outcomes of all students, focusing on their presumed knowledge rather than strictly what has been taught These tests are closely aligned with the teaching content and specific objectives In this context, we will review the grammar and vocabulary topics covered this semester, as well as the assessments conducted in parts 1, 2, and 3.
Unit What they have been taught What they have been tested Unit 7 Past simple tense
Unit 9 Count and uncount nouns
Question1, 4 (part 1) Question 2 (part 2) Question 5 (part 2)
Question 2(part 1) Question 3 (part 3) Question 5 (part 3) Unit 14 Present perfect
Present perfect and past simple Question 3 (part 2)
Table 4: What students had been taught and what they had been checked in part I, II, III of the test
And what students have been taught in writing part
Table 5: What students had been taught and checked in the writing part
The test content is generally adequate, featuring clear instructions and familiar grammar structures taught throughout the semester However, certain grammatical elements, such as the negative form of the past simple from Unit 8 and "going to" from Unit 11, were not assessed Although the writing section related closely to Unit 10, students faced challenges expressing their ideas in English without guidance, which hindered their ability to write coherent paragraphs Consequently, while the writing section reflected students' learning, it failed to demonstrate their progress effectively, leading to concerns about its content validity Additionally, the subjective nature of the writing assessment posed difficulties for teachers in marking.
3.3.2 Analysis of survey questionnaire for the second year students
The survey questionnaire includes seven questions focused on students' feedback regarding various aspects of the final achievement test, such as time, content, and format Its aim is to gather insights that will inform the next chapter, ensuring that the final achievement test achieves high content validity.
I delivered the survey questionnaire (see in Appendix 3) to 65 students; the results of the survey questionnaire are analyzed as follows: a Question 1: Does the test measure what you have been taught?
Chart 1: Students' comment on the validity of the test
According to the survey results, 80% of students believe that the current achievement test effectively measures their language abilities, reflecting what they have learned In contrast, 20% disagree, arguing that the test items primarily derive from the course and workbooks, making it possible for anyone who studies these materials to achieve high marks They suggest that the test should incorporate questions beyond the standard curriculum to better differentiate student performance and encourage greater effort in learning While the final achievement test is deemed suitable for all student populations, there is a need to restructure it by including more challenging items to motivate high-performing students.
Chart 2: Students' comment on time allowance of the test
17 % 3 % enough not enough too much
About 3% of students felt that the test duration was excessive, while 80% reported no issues with time management Conversely, 17% indicated they needed more time to complete the test Overall, this suggests that the majority of students found the allotted time sufficient for finishing the test.
Chart 3: Students' comment on difficult level of the test
14 % very difficult difficult average easy
A survey of students regarding the difficulty level of a recent test revealed varied opinions: 23% found the test difficult, making it challenging to achieve high marks, while 58% considered it average, as they were able to score well in the English test Conversely, 14% deemed the test easy, completing it in two-thirds of the allotted time, and only 5% felt it was very hard Despite these differing views, all students agreed that writing a paragraph is one of the most challenging aspects of the test, often requiring significant time to generate and organize their ideas logically, especially without any provided structure or prompts.
The test is designed to be of average difficulty, allowing most students to complete it within the allotted time, making it appropriate for their skill level Additionally, it is important to consider whether all types of questions on the test are familiar to the students.
Almost all students reported that they easily understand the test instructions and have practiced similar test formats both in class and at home Additionally, they confirmed that they have been taught the necessary grammar structures and vocabulary relevant to parts 1, 2, and 3 of the test.
A survey revealed that 58% of students found the grammar structure and vocabulary used in their studies to be appropriate and relevant, while 42% felt it was disconnected from their lessons These students attributed their struggles to a lack of attention in studying and revising the material, leading to uncertainty about their retention of the grammar and vocabulary learned Additionally, Question 6 focused on the topic of the writing section.
A significant majority of students, 85%, expressed that the writing topic was relevant and aligned with their lessons, while 15% selected a different response.
Many students struggle to write coherent paragraphs due to a lack of guidance and understanding of how to express their ideas in English They often find it challenging to organize their thoughts effectively, which hinders their ability to create complete and structured paragraphs.
The writing section of the test demonstrated the students' knowledge but failed to accurately reflect their progress in writing skills Consequently, the content validity of this section was deemed unsatisfactory.
Chart 4: Result of the test
The test results indicate a concerning trend, with 69% of students scoring between 1.5 and 5, while only 31% achieved marks ranging from 5 to 8 Many students expressed difficulty in grasping the material presented by their teachers, and those who did understand often forgot the content quickly due to insufficient practice and revision of the lessons taught.
3.3.3 Analysis of the survey questionnaire for teachers
Results
In conclusion, the test results indicate that the students' learning aligns with the content from "New Headway Elementary - The Third Edition," achieving several objectives of the new textbook Survey responses from students largely support this correlation, while teacher feedback confirms that the grammar structures and vocabulary taught were fully integrated into the test content.
The analysis of the final achievement test provided insights into its content validity, while questionnaires served as an effective tool for evaluation By gathering feedback from both teachers and students, the writer was able to conduct a thorough assessment of the test, identifying its strengths and weaknesses.
The test is designed with a valid format and clear instructions, ensuring its effectiveness It evaluates students on their understanding of key areas, including grammar structures, vocabulary, and knowledge about the capital city.
The test demonstrated content validity by aligning with the objectives and material outlined in the "New Headway Elementary - Third Edition" syllabus, indicating that certain aspects of the test effectively reflected the intended curriculum.
Upon reviewing the course book, it becomes evident that the test did not adequately assess all the grammar and vocabulary covered, such as the negative form of the past simple tense in Unit 8 and the "going to" structure in Unit 12.
Certain questions were overly simplistic, failing to accurately reflect students' understanding and merely assessing their recall abilities Additionally, the writing section lacked guiding sentences or suggestions, which left students without a clear framework to develop their ideas effectively.
The syllabus and textbook outline four essential language skills; however, the final achievement test focused solely on writing, neglecting to assess listening and reading skills This oversight raised concerns about the validity of the test results in certain instances.
A survey of teachers revealed a significant issue with the test, indicating that the designers lacked clear specifications for the skills and structures it was intended to assess To enhance the effectiveness of the final achievement test, it is recommended that a test specification grid be developed.
Recommendations and conclusions 4.1 Recommendations
Conclusion
This study examined the content validity of the final achievement test for second-year students at Phuong Dong University Through careful analysis of data collected from two sets of survey questionnaires completed by teachers and students, along with an evaluation of the final achievement test itself, the research addressed two key questions.
The analysis reveals that the test lacks content validity, as it primarily assesses only a limited range of grammatical structures and vocabulary, without evaluating essential skills like reading and listening Additionally, it overlooks important grammatical concepts taught to students, such as the negative form of the past simple tense and the "going to" structure Furthermore, the writing section, identified as the most challenging part of the test, provides no guiding sentences or questions, making it difficult for students to complete Consequently, it can be concluded that the content validity of this test is inadequate.
Based on a comprehensive literature review and document analysis, the author offers recommendations for enhancing the design of final term tests The focus is on creating a suggested test specification that teachers can use to develop more valid assessments, ultimately improving the quality and effectiveness of the testing process.
Limitations
This study, despite the utmost efforts of the writer, remains some limitations First of all, the study can not cover all the final achievement tests for second year students
Secondly, the scope of this study is limited to second year students at Phuong Dong University
The study acknowledges limitations in the data collection methods used for students and teachers, particularly in the survey questionnaires discussed in chapter three Due to constraints in time and resources, the research could not explore additional qualities of the test, highlighting the need for further investigation in this area.
The author aims for this minor thesis to be positively received by both teachers and students at Phuong Dong University, with the hope that it will contribute, even in a small way, to enhancing the teaching and learning of English at the institution.
In this chapter, the author outlines recommendations for enhancing the final term test design By implementing these suggestions, educators can create a more valid and effective assessment compared to previous versions.
1 Alderson, J.C., C Clapham & D Wall (1995) Language test construction and evaluation Cambridge: Cambridge University Press
2 Anastasi, A (1982) Psychological Testing London: Macmillan
3 Allen, J.P.B and Corden, Pit, S (1974) Techniques in Applied Linguistics, Oxford: Oxford University Press
4 Bachman, L.F (1990) Fundamental considerations in Language testing Oxford:
5 Bachman, L.F., and Palmer, A.S (1996) Language testing in practice: designing and developing useful language tests Oxford: Oxford University Press
6 Bachman, L.F.et al (1995) An investigation into the comparability of two tests of English as a foreign language Cambridge: Cambridge University Press
7 Brown, H D (1994) Principles of Language Learning and Teaching, 3rd edn,
8 Brown, H D (2000) Principles of Language Learning and Teaching New York: Longman
9 Brown, J D (1997) Statistics Corner: Questions and answers about language testing statistics: Reliability of surveys Shiken: JALT Testing & Evaluation SIG
Newsletter, 1(2), 17-19 Also retrieved from the World Wide Web at http://www.jalt.org/test/bro_2.htm
10 Brown, J D (1990) Where do tests fit into language programs? JALT Journal, 12(1), 121-140
11 Carroll, J.B (1968) The psychology of language testing (In) A David (Ed) Language Testing Symposium A psycholinguistic Perspective London: Oxford University Press
12 Guilford, J.P (1965) Fundamental Statistics in Psychology and Education, 4th Edn New York: McGraw-Hill
13 Harrison, A (N.D) A Language Testing Handbook Modern English Publications
14 Henning, G (1987) A guide to Language testing Cambridge: Newbury House Publisher
15 Hughes, A (1988) Writing English Tess (2 nd Edition) London: Longman
16 Hughes, A (1989) Testing for Language Teachers Cambridge: Cambridge
17 Ibe, M D (1981) Language test analysis beyond the validity and reliability criteria (In) Papers on Language Testing Occasional Papers Read J.A.S (Ed)
No 18.pp1 Singapore: SEAMEO Regional Language Center
18 Kelly, R (1978) On the construct validity of comprehensive tests: an exercise in applied linguistics University of Queensland PhD thesis
19 Kerlinger, F N (1973) Foundations of behavioral research New York: Holt, Rinehart, & Winston
20 Moller, A.D (1982) A study in the validation of proficiency tests of English as a Foreign Language Unpublished PhD thesis, University of Edinburgh
21 Moore, G E (1992) The significance of research in vocational education: 1992
AVERA presidential address The Journal of Vocational Education Research, 17(4),
22 Oller, J.W Jr (1979) Language tests at school London: Longman
23 Read, J.A.S (1983) What is a good classroom test? (In) Guidelines A Periodical for Classroom Language Teachers Classroom tests Crabbe, D (Ed.) Volume 5
No.1.pp1-7.Singapore: SEAMEO Regional Language Center
24 http://www.cis.doshisha.ac.jp/kkitao/library/article/test/design2.htm
Appendix 1 The content of the course book
Contents Language Input Skills development
Using a bilingual dictionary Plural nouns
The family Opposite adjectives Food and drink
In a cafe An email from England
Verbs help, make, serve Jobs
Seumas Mc sopporran – the man with 13 jobs
Asking about a friend and relatives
Verbs: relax, go out, play Leisure activities
Leisure activities What is your favorite season?
A questionnaire – Do you have a healthy lifestyle?
My favorite season Where are they?
Rooms Things in the house
What is in your picture?
What is in Yoshi’s briefcase?
What is in your bag?
Countries and language Verbs Words that sound the same
Formal letters Applying for a job
Verb Words that go together
The year I was born What did it happen?
Relationships Spelling and silent letters Phonetic symbols
How did you two meet?
Three inventions How did you two meet
Food you like Role play- shopping Meals in your country
City and country adjectives City and country words
Directions 2 Viva la danza- Havana/
I have got more than you
A walk in the country/city
Comparing life in the city and country
Clothes Describing people Words that rhyme
Describing a person/ scene Getting information- who is at the party?
Who is at the party?
Born free Dangerous sports Interviews World weather
Present perfect and past simple
We’ ve never learnt to drive
Cities you have been to Things have been done
This questionnaire aims to enhance the quality of the final achievement test for non-major students Your honest and precise responses are crucial for this improvement.
1 Does the test measure what you have been taught? a Yes b No
2 In your opinion, time allowance for this test is… a Not enough b Enough c Too much d Your own ideas
3 What do you think about the difficulty level of each part in the test? a very difficult b difficult c average d easy
4 Are all types of the test familiar with you? a Yes b No
5 Have you been taught grammatical structures and vocabulary in part 1,2 and 3 of the test? a Yes b No
6 The topic of the writing part is a Appropriate and relevant b Not related to what you have been taught
7 Which mark did you get for the final achievement test?
1 In your opinion, what is the purpose of the test? a To assess teachers’ teaching effectiveness c To classify or rank the students b To measure the students’ ability d To evaluate the effectiveness of a book
2 What did the teacher base on when designing the final achievement test for second year non major students? a Level of the students b The teaching and learning syllabus c The teacher’ book
3 Does the test measure what students have been taught? a Yes b No
4 Are all types of the test similar to what the students have been taught? a Yes b No
5 Have all the grammatical structures and words in the test been taught? a Yes b No
(If not, what grammatical structures and words aren’t taught?………
6 Is the topic of the writing part relevant to what students have been taught? a Yes b No
7 Please evaluate the difficulty level of each part in the test to the level of students
8 In your opinion, time allowance for this test is… a Not enough b Enough c Too much
(If too much or not enough, how much time do you think is appropriate for the test?
9 Before giving the test to students, was it discussed among teachers to find out its content validity? a Yes b No
Appendix 4 Answer key for reading task
1 Because he played cards for hours and got very hungry
2 in the oven of a stove
5 The cook cut some very thin pieces of potatoes and fried in oil Then he put salt on them
II Decide the following sentences true (T) or false (F)
Appendix 5 Answer key for new final achievement test
C: Our house is quite old, bout fifty years old It is in the centre of the village near the church All the houses here are white We have a living room, quite a big kitchen, and three bedrooms, and a big verandah all around the house
B: Our children are not at home now They both have jobs in the city so most of the time it is just Candy and me
C: Yes, so in summer we do bed and breakfast for tourists We have lovely visitors from all over the world
When I grow up, I am going to be a footballer- a really good one I am in the school team and I play three times a week But I am going to train very hard, everyday so I can be really, really good First, I am going to play for Manchester United, then Inter Milan, and then Real Madrid Those are my favorite teams I am going to travel all over the world and I am going to be famous I am not going to marry until I am very old- about
25 Then I want to have two sons I am going to play football until I am 35- that is very long time And I am going to teach my sons to play I want them to be famous footballers, too