Introduction 1.1 Rationale
Scope of study
This thesis focuses on evaluating the content validity of a final achievement test by comparing its objectives, syllabus, and textbook allocation with the actual test contents The study analyzes data from the currently utilized test and offers practical recommendations for its improvement.
Due to constraints in time, resources, and capabilities, the writer is unable to address all tests comprehensively; instead, only select recommendations for test improvements are provided.
Aims of study
This study evaluates the content validity of the final achievement test designed for second-year non-major students at Phuong Dong University, focusing on a thorough analysis of the test's content.
The specific aims of this research are:
- To find out the strengths and weaknesses of the currently used test with reference to the content validity
- To suggest some improvements for the test.
Methods of study
In order to achieve the above-mentioned aims, a combination of many methodologies was utilized
The author grounded her analysis in language testing theories and principles, emphasizing the importance of content validity in effective assessments By reviewing various reference materials, she established a theoretical framework to evaluate the achievement test administered to second-year students, specifically assessing its alignment with the curriculum covered in their first semester This examination aims to determine the test's content validity based on the knowledge and skills students have acquired.
Qualitative methodologies were utilized, employing survey questionnaires to gather data from English teachers and students at Phuong Dong University Two sets of questionnaires were distributed to assess their evaluations of the content validity of the final achievement test and to gather suggestions for its enhancement.
Research questions
In this study, the writer tries to answer the two following questions:
Question 1: What are the strengths and weaknesses of the final achievement test with reference to the content validity for second year non major students at Phuong Dong University?
Question 2: What are some suggested solutions for the improvements of the test?
Design of study
The thesis is organized into five major chapters:
1 Chapter 1 INTRODUCTION presents such basic information as: the rationale, the aims, the methods, the research questions and the design of the study
2 Chapter 2 LITERATURE REVIEW presents a review of related literature that provides the theoretical basis for evaluating and building a good language test This review includes background on language testing, criteria of good tests and theoretical issues on test content validity
3 Chapter 3 THE STUDY mentions the methods used in the research and which shows its detailed results of the surveys including the questionnaires and the analysis of the final achievement test in order to find out its problems with reference to the content validity
4 Chapter 4 RECOMMENDATIONS AND CONCLUSIONS Recommendations provide some suggestions for the improvements of the final achievement test basing on the mentioned theoretical and practical study Conclusions summarize the matters of research, its findings as well as its limitations.
Literature review
The roles of language testing
Language testing is a form of measurement It helps the teachers:
Evaluating a learner's achievement in a language program involves assessing their language knowledge based on the curriculum and materials covered during the course.
Evaluating a learner's language proficiency is essential for determining their suitability for future language use, such as assessing whether an individual possesses the necessary skills to become a tourist guide This evaluation focuses on the practical application of language skills, independent of the specific programs or materials the learner has utilized.
+ To diagnose a learner‟s strengths and weaknesses in a language and to attempt to explain why the certain problems occur and what treatments could be used to tackle these problems
+ To classify or place the testees in the appropriate language classes
+ To measure the testee‟s aptitude for learning a language
To assess the effectiveness of a language program, researchers typically utilize experimental and control classes that share the same educational goals but employ distinct methods and materials to reach those objectives, as noted by Brown (2000:5).
Bebecca M Valette (1977) highlights three crucial roles of classroom tests in second language teaching: they help define course objectives, stimulate student progress, and evaluate overall class achievement.
Classroom tests play a crucial role in defining course objectives, as students tend to adapt their study habits based on the types of assessments they encounter Despite a teacher's emphasis on oral fluency, if the assessments primarily consist of written tests, students are likely to focus on improving their reading and writing skills instead.
Tests play a crucial role in stimulating student progress by providing a rewarding experience that allows students to demonstrate their proficiency in the target language Unlike traditional assessments aimed at highlighting student shortcomings, modern tests should be announced in advance, enabling adequate preparation Prompt feedback after testing is essential, as it helps students understand their performance, reinforcing correct answers and addressing errors This approach ensures that tests are integral to the learning process, fostering a positive and constructive educational environment.
Testing plays a crucial role in assessing student achievement and identifying challenges within the curriculum By conducting regular assessments, teachers can pinpoint specific areas where individual students and the class may struggle Analyzing test results allows educators to focus on targeted drills and provide tailored support for each student Additionally, testing serves as a valuable tool to evaluate whether class objectives are being met and to assess the effectiveness of various teaching methods, approaches, or materials Ultimately, classroom tests provide an objective measure of each student's progress, highlighting their attainment of course goals and performance relative to their peers.
Relationship between testing and teaching- learning
Historically, teaching and testing have been viewed as separate entities, both theoretically and practically According to Williams (1983), testing is often an unpleasant necessity outside the classroom for two primary reasons Firstly, testing emphasizes competition over cooperation; while classroom activities encourage collaboration through pair and group work, tests require individual performance, which can obscure the true abilities of group members Secondly, testing creates a dichotomy of winners and losers, where those who perform well may not feel significant pressure, but those who struggle can experience feelings of inadequacy This competitive nature not only hinders student collaboration but also limits the supportive relationship between teachers and learners during assessments.
In recent years, a significant trend in English testing has emerged, focusing on integrative and communicative assessments that introduce innovative techniques Researchers widely acknowledge the strong connection between teaching and testing, highlighting the importance of aligning assessment methods with educational practices.
Teaching and testing are intricately linked, making it challenging to separate them (1994) Tests serve not only to assess student performance but also to reinforce learning and motivate students Essentially, tests act as an extension of classroom activities, offering valuable insights that enhance both teaching and learning Additionally, effective teaching and learning generate rich language materials that can be utilized in testing.
A well-designed test serves as an essential educational tool by providing teachers with insights into the effectiveness of their instruction It enables educators to assess students' abilities and understand individual characteristics Additionally, tests allow for ongoing monitoring and evaluation of student learning, helping to identify strengths and weaknesses in real-time Ultimately, test results inform teachers about the effectiveness of their syllabus, teaching methods, and materials, ensuring continuous improvement in the educational process.
Testing can significantly impact teaching and learning, both positively and negatively When a test is deemed crucial, it often overshadows other educational activities, leading teachers to concentrate solely on test content This focus can distort the teaching program, prioritizing test preparation over a well-rounded education.
2.3 Major characteristics of a good test
Before creating a test, it's essential to consider the key characteristics that define a quality assessment According to Harrison (1983: 10), a good test must possess four fundamental traits: validity, reliability, practicality, and discrimination These characteristics ensure that the test effectively measures what it intends to, produces consistent results, is feasible to administer, and can differentiate between varying levels of performance.
Validity is a crucial characteristic of effective testing, and it has been a topic of debate for many years Recently, discussions in language testing have shifted towards viewing validity as a comprehensive concept that encompasses various types, highlighting its multifaceted nature.
Henning (1987:5) defines validity as follows:
Validity refers to how well a test or its components measure what they are intended to measure A test is considered valid if it accurately assesses the specific construct it claims to evaluate Therefore, when discussing a test's validity, it is important to note that a test may be valid for certain purposes while not being valid for others.
Test validity refers to the extent to which a test accurately measures what it is designed to assess For instance, a listening test with challenging written multiple-choice options may inadvertently evaluate reading comprehension, particularly affecting students who excel in listening but struggle with reading Validity is determined by how well the test aligns with its intended purpose; for example, a test designed to evaluate English communication skills is valid only if it effectively assesses that ability Moreover, validity exists on a spectrum, meaning some tests are more valid than others, and tests can only be deemed valid or invalid based on their specific intended uses If a test aims to measure reading skills but also assesses writing, it may not be valid solely for reading but could still evaluate both skills together.
Validity pertains to the accuracy of conclusions drawn from test results regarding individuals and groups It is essential to evaluate the correctness of specific inferences related to test takers, making the measurement of validity a complex task.
Validity in research encompasses various types, including face validity and content validity, which are crucial for ensuring the accuracy and relevance of measurements This article will specifically explore these two primary forms of validity, emphasizing their significance in the evaluation process.
Face validity refers to the extent to which a test appears, at first glance, to measure what it is intended to measure from the learners' perspective It is primarily assessed through the content of the test; if the test reflects the actual skills or knowledge that learners have or aim to acquire, it is considered to have face validity As noted by Arthur Hughes (1989), a test is deemed to possess face validity if it visibly aligns with its intended measurement goals For instance, a test claiming to assess pronunciation skills but not requiring spoken responses may be viewed as lacking face validity Tests without face validity are often rejected by candidates, educators, and educational authorities Ultimately, face validity is influenced by the perceptions of non-experts, including candidates, their families, and the public, and can be gauged through feedback from other educators.
The rise of communicative language testing has led to a greater focus on face validity, emphasizing the importance of tests resembling real-world language use This connection to "real life" enhances the perceived validity of the test While students' opinions may not be expert, they provide valuable insights, as test takers' perceptions can influence their performance If a test seems invalid to the participants, they may not put forth their best effort, highlighting the significance of understanding non-expert perspectives in the testing process.
In other words, the face validity affects the response validity of the test This critical view of face validity provides a useful method for language test validation
Face validity serves as a quick and effective guide, balancing concerns about statistical analysis in testing When a test possesses strong face validity, it helps maintain student motivation; conversely, a test perceived as irrelevant by students will lack face validity It is possible for a test to encompass all components of a teaching program while still lacking face validity Although the concept of face validity is not new in language testing, its current emphasis is relatively recent Historically, many test creators viewed face validity merely as a public relations tool, but today, designers of communicative tests consider it the most crucial aspect of test validity.
Among several kinds of validity, the simplest and most important one to the language teachers is content validity
According to Read (1983:6), content validity is the most crucial type of validity for classroom testing, as it ensures that the test reflects the syllabus's content and objectives To accurately assess students' progress, the test must include a representative sample of the knowledge, skills, or functions they are expected to master Therefore, focusing on the test's content is essential for achieving content validity.
Kerlinger (1973) defines content validity is the representative or sampling adequacy of the content, the substance, the matter and the topics of a measuring instrument
In the same way, Harrison (1983: 11) defines content validity as:
Major Characteristics of a good test 1 Test validity
3.1 English learning, teaching and testing at Phuong Dong University 3.1.1 The students
Phuong Dong University attracts students from various regions across the country, many of whom have limited experience with English due to their focus on subjects like mathematics, physics, and chemistry during high school to prepare for university entrance exams As a result, these students often enter university as true beginners in English, exhibiting a range of language proficiency levels.
English teachers instructing second-year students vary in age, with half ranging from 45 to 55 years old and the other half between 25 and 38 years old These educators have graduated from three prominent institutions: Ha Noi National University, Ha Noi Foreign Language University, and Phuong Dong University.
3.1.3 The course book: “New Headway Elementary- The third edition”
The textbook "New Headway Elementary, Third Edition" is utilized for teaching second-year students at Phuong Dong University, catering specifically to those at the elementary level.
It consists of 14 units, designed in a harmonious combination with powerful lexical to increase learners‟ vocabulary and develop awareness of the English culture
Each unit consists of three sections that emphasize grammar, function, or vocabulary Students are given opportunities to enhance their understanding in grammar, vocabulary, communication skills, and pronunciation through various practice activities, including listening, speaking, reading, and writing.
The study 3.1 English learning, teaching and testing at Phuong Dong University 3.1.1 The students
The teachers
English teachers instructing second-year students are diverse in age, with half ranging from 45 to 55 years old and the other half between 25 and 38 years old These educators have graduated from three notable institutions: Ha Noi National University, Ha Noi Foreign Language University, and Phuong Dong University.
The course book “New Headway Elementary- The third edition”
The third edition of "New Headway Elementary" serves as the primary textbook for second-year students at Phuong Dong University, specifically tailored for those at the elementary level of English proficiency.
It consists of 14 units, designed in a harmonious combination with powerful lexical to increase learners‟ vocabulary and develop awareness of the English culture
Each unit consists of three sections, concentrating on grammar, function, and vocabulary Students engage with various practice activities in listening, speaking, reading, and writing, enhancing their skills in grammar, vocabulary, communication, and pronunciation.
Syllabus and its objectives
In the first semester of their second year, students will cover seven units (units 7 to 14) over 45 periods, each lasting 50 minutes, within approximately 9 weeks The curriculum focuses on enhancing grammar, vocabulary, communication skills, and pronunciation while allowing students to engage with various topics The primary objectives of the course are to improve students' foundational knowledge in vocabulary and grammar, as well as to practice the four essential language skills: listening, speaking, reading, and writing in social contexts.
The final achievement test
The final achievement test consists of the following parts: types, items, tasks
Part Types Items Tasks Marks
Part 1 Rewrite the sentences 5 Rewrite sentences so that there is no change of meanings
5 Use the following sets of words to write complete sentences
Part 3 Correct mistakes 5 Find and correct one mistake in each sentence
Part 4 Write a paragraph 1 Write a paragraph of 100-120 words about your capital city
Table 3: The components of the final achievement test
The marking criteria for the test have caused confusion among teachers and anxiety among students Specifically, part 4 presents challenges for educators due to the lack of detailed assessment guidelines regarding language, content, and grammar.
Research method
This study employs both quantitative and qualitative methods, specifically survey questionnaires and document analysis, with a primary focus on document analysis to evaluate the strengths and weaknesses of the final achievement test in terms of content validity Additionally, survey questionnaires are utilized to gather insights from both teachers and students regarding the test While each method contributes to the collection and validation of diverse data types, they also possess inherent limitations.
Survey questionnaires are an effective method for data collection, particularly for assessing the attitudes and opinions of teachers and students regarding the content validity of end-of-term tests One key advantage is that they facilitate a non-confrontational environment, allowing informants to freely express their thoughts through a structured list of questions Additionally, the use of primarily closed-ended questions simplifies the data collection and analysis process Furthermore, this method can yield a large volume of responses, enhancing the overall reliability of the findings.
Besides survey questionnaires, document analysis is considered as the main method to evaluate the final achievement test in terms of the content validity
This article will examine "The New Headway - Elementary - Third Edition" to identify the teaching requirements for educators and the learning expectations for students The primary aim of this study is to evaluate the content validity of the final achievement test for second-year students at Phuong Dong University Analyzing this test serves as an effective method to achieve this goal By applying testing theories, principles of test design, and the characteristics of a quality assessment, the writer will compare the course objectives and the students' learning outcomes with the test content This analysis will highlight the strengths and weaknesses of the test, leading to recommended improvements.
Last but not least, the writer will analyze the data of survey questionnaires from both teachers and students to see how their comments about this test are
Utilizing multiple methodologies is crucial for validating results and ensuring authenticity in research By incorporating document analysis alongside survey questionnaires, researchers can capture genuine feelings and perspectives of informants, providing a comprehensive view of the actions and experiences of teachers and students This combined approach yields objective and reliable findings, enhancing the overall quality of the study.
Data analysis
This section will analyze the final achievement test by comparing its content with the material covered by students in the first semester to identify the strengths and weaknesses of the current test in terms of content validity Additionally, the opinions of both students and the teacher, gathered through survey questionnaires, will be examined to provide a more comprehensive evaluation of the test's content validity.
3.3.1 Analysis of the final achievement test
The final achievement test for second-year non-major students must be carefully evaluated in terms of both its layout and content The test is structured into four key components that are essential for assessing student performance effectively.
The final achievement test - No1
Time allowed: 60 minutes Total score Marker’s signature1:
I Rewrite each sentence, beginning as shown, so that the meaning stays the same
1 My watch is cheaper than yours
2 Mick is a very careful driver
3 What are your favorite things at the weekend?
4 No one is more intelligent than Anna in her class
5 Do you want some fish and chips?
II Guided sentence building: use the following sets of words and phrases to write complete sentences
1 I‟d/ chicken/and chips/main course
2 Everest/highest/mountain/the world?
3 You/been/Machu Pichu/before?
4 Alice/tall/long black hair/blue eyes
5 My sister/have/new car/but/drive/it
III Find and correct ONE mistake in each of the following sentences
1 My brother can play badminton when he was five years old
2 Who‟s is this jacket on your chair?
3 This morning, I had breakfast quick because I was late for school
4 It is good to see you Where have you gone?
5 Martin is very good at badminton He won the game easy
IV Writing topic: Write a paragraph of 100-120 words about your capital city
The final achievement test is a formal assessment designed to evaluate the overall performance of all students, focusing on presumed learning rather than actual learning or teaching It is essential that the test content aligns closely with the teaching materials and objectives In this context, we will review the grammar and vocabulary topics covered with students this semester, as well as the assessments conducted in parts 1, 2, and 3.
Unit What they have been taught What they have been tested Unit 7 Past simple tense
Unit 9 Count and uncount nouns
Question1, 4 (part 1) Question 2 (part 2) Question 5 (part 2)
Infinitive of purpose Unit 13 Question forms
Question 2(part 1) Question 3 (part 3) Question 5 (part 3) Unit 14 Present perfect
Present perfect and past simple Question 3 (part 2)
Table 4: What students had been taught and what they had been checked in part I, II, III of the test
And what students have been taught in writing part
Table 5: What students had been taught and checked in the writing part
The test content is adequately structured with clear instructions, utilizing familiar vocabulary and grammar previously taught during the semester However, it fails to assess certain grammatical aspects, such as the negative form of the past simple from Unit 8 and the "going to" structure from Unit 11 While the writing section aligns with Unit 10's teachings, students struggle to articulate their thoughts in English without guidance, hindering their ability to write coherent paragraphs Consequently, the writing assessment does not effectively reflect students' progress, leading to concerns about its content validity Additionally, the subjective nature of grading this section poses challenges for teachers.
3.3.2 Analysis of survey questionnaire for the second year students
The survey questionnaire comprises seven questions focused on students' feedback regarding various aspects of the final achievement test, including its time, content, and format The primary aim of this survey is to gather insights that will inform the next chapter, ultimately enhancing the content validity of the final achievement test.
I delivered the survey questionnaire (see in Appendix 3) to 65 students; the results of the survey questionnaire are analyzed as follows: a Question 1: Does the test measure what you have been taught?
Chart 1: Students' comment on the validity of the test
According to the survey, 80% of students believe that the current achievement test effectively measures their language abilities, reflecting what they have learned In contrast, 20% disagree, arguing that the test items primarily stem from the course and workbooks, making it easy for those who study these materials to achieve high marks They suggest that tests should include content beyond the provided materials to better assess student capabilities and encourage deeper learning While the final achievement test is deemed suitable for all students, it requires restructuring to incorporate more challenging items that would motivate high-performing students.
Chart 2: Students' comment on time allowance of the test
17 % 3 % enough not enough too much
A survey revealed that around 3% of students felt the test duration was excessive, while 80% reported no issues with time management Conversely, 17% indicated they needed more time to complete the test Overall, these findings suggest that the majority of students felt adequately timed to finish the test.
Chart 3: Students' comment on difficult level of the test
14 % very difficult difficult average easy
A recent survey revealed varied opinions among students regarding the difficulty of their English test While 23% of students found the test challenging, 58% considered it to be of average difficulty, allowing them to achieve high marks Conversely, 14% deemed the test easy, completing it in two-thirds of the allotted time, while a small group of 5% struggled significantly Despite these differences, all students agreed that writing a paragraph remains one of the most challenging aspects, often requiring considerable time to generate and organize their ideas logically, particularly in the absence of a structured outline.
The test is of average difficulty, allowing most students to complete it within the allotted time, making it suitable for their level Additionally, it is important to assess whether all types of questions on the test are familiar to the students.
Almost all students reported that they easily understand the test instructions and have practiced similar test formats both in class and at home Additionally, they have received instruction on grammar structures and vocabulary relevant to parts 1, 2, and 3 of the test.
A recent survey revealed that 58% of students found the grammar structure and vocabulary used in their studies to be appropriate and relevant In contrast, 42% expressed that these elements did not align with their lessons, attributing their confusion to a lack of attention in studying and revising the material Consequently, they are unsure whether they have fully grasped the grammar and vocabulary taught Additionally, Question 6 of the survey inquired about the topic of the writing section.
A significant majority of students, 85%, confirmed that the writing topic was relevant and aligned with their lessons, while 15% expressed a differing opinion.
Students often struggle with writing coherent paragraphs due to a lack of understanding in expressing their ideas in English and organizing them effectively Without guidance, they find it challenging to articulate their thoughts clearly and structure their writing appropriately.
The writing assessment demonstrated the students' learning; however, it failed to accurately reflect their progress in writing skills, indicating that the content validity of this section was inadequate.
Chart 4: Result of the test
The test results indicated a concerning trend, with 69% of students scoring between 1.5 and 5, while only 31% achieved marks between 5 and 8 Many students reported difficulties in understanding the material presented by their teachers, and those who did grasp the concepts often struggled to retain the information due to insufficient practice and revision.
3.3.3 Analysis of the survey questionnaire for teachers
Results
In conclusion, the test effectively reflected the content covered in "New Headway Elementary - The Third Edition," indicating that several objectives of the new textbook were met Most students, as revealed through questionnaires, supported this assessment, while teacher informants confirmed that the grammar structures and vocabulary were fully aligned with the material taught.
The analysis of the final achievement test allowed for a thorough examination of its content validity, while questionnaires proved to be an effective tool for evaluation By gathering insights from both teachers and students, the writer was able to conduct an in-depth assessment of the test, identifying its strengths and weaknesses in the current educational context.
The test is valid and features clear instructions, effectively assessing students' understanding of grammar structures, vocabulary, and knowledge about the capital city.
The test demonstrated content validity by aligning with the objectives and materials outlined in the "New Headway Elementary - The Third Edition" syllabus, indicating that it effectively assessed the relevant knowledge and skills in certain areas.
Upon reviewing the course book, it becomes evident that the test did not adequately assess all the grammar and vocabulary presented Notably, it overlooked the negative form of the past simple tense discussed in Unit 8 and the "going to" structure covered in Unit 12.
Certain questions in the assessment were overly simplistic, failing to accurately reflect students' understanding of the material and merely testing their recall abilities Additionally, the writing section lacked guiding sentences or suggestions, leaving students without a clear framework to develop their ideas.
The syllabus and textbook outline four essential language skills; however, the final achievement test only assessed writing skills This oversight resulted in the neglect of listening and reading skills, leading to concerns about the validity of the test in certain instances.
A significant issue identified through teacher surveys is that the test designers lacked clear specifications for the skills and structures intended to be assessed To enhance the effectiveness of the final achievement test, creating a detailed test specification grid is recommended.
Recommendations and conclusions 4.1 Recommendations
Conclusion
This study examined the content validity of the final achievement test for second-year students at Phuong Dong University Through a detailed analysis of data gathered from two survey questionnaires targeting both teachers and students, as well as an evaluation of the final achievement test itself, the research effectively addressed two key questions.
The analysis indicates that the test lacks content validity, as it primarily focuses on grammatical structures and vocabulary without assessing reading and listening skills Additionally, it overlooks essential grammar concepts taught to students, such as the negative form of the past simple tense and the "going to" structure The writing section, the most challenging part of the test, provides no guiding sentences or questions, making it difficult for students to complete Consequently, it can be concluded that the content validity of this test is unsatisfactory.
Based on a comprehensive literature review and document analysis, the author offers recommendations for enhancing the design of final term tests Emphasizing the importance of validity, the writer proposes a suggested test specification aimed at guiding teachers in creating more effective assessments for their students.
Limitations
This study, despite the utmost efforts of the writer, remains some limitations First of all, the study can not cover all the final achievement tests for second year students
Secondly, the scope of this study is limited to second year students at Phuong Dong University
The data collection methods used for students and teachers, including survey questionnaires, have notable limitations as discussed in Chapter Three Due to constraints in time and resources, this study could not explore additional aspects of the test's quality, indicating a need for further research in this area.
The author expresses the hope that this minor thesis will be positively received by both teachers and students at Phuong Dong University, contributing to the enhancement of English teaching and learning at the institution.
In this chapter, the author offers recommendations for enhancing the final term test's design By implementing these suggestions, educators can create a more valid assessment compared to previous versions.
1 Alderson, J.C., C Clapham & D Wall (1995) Language test construction and evaluation Cambridge: Cambridge University Press
2 Anastasi, A (1982) Psychological Testing London: Macmillan
3 Allen, J.P.B and Corden, Pit, S (1974) Techniques in Applied Linguistics, Oxford: Oxford University Press
4 Bachman, L.F (1990) Fundamental considerations in Language testing Oxford:
5 Bachman, L.F., and Palmer, A.S (1996) Language testing in practice: designing and developing useful language tests Oxford: Oxford University Press
6 Bachman, L.F.et al (1995) An investigation into the comparability of two tests of English as a foreign language Cambridge: Cambridge University Press
7 Brown, H D (1994) Principles of Language Learning and Teaching, 3rd edn,
8 Brown, H D (2000) Principles of Language Learning and Teaching New York: Longman
9 Brown, J D (1997) Statistics Corner: Questions and answers about language testing statistics: Reliability of surveys Shiken: JALT Testing & Evaluation SIG
Newsletter, 1(2), 17-19 Also retrieved from the World Wide Web at http://www.jalt.org/test/bro_2.htm
10 Brown, J D (1990) Where do tests fit into language programs? JALT Journal, 12(1), 121-140
11 Carroll, J.B (1968) The psychology of language testing (In) A David (Ed) Language Testing Symposium A psycholinguistic Perspective London: Oxford University Press
12 Guilford, J.P (1965) Fundamental Statistics in Psychology and Education, 4th Edn New York: McGraw-Hill
13 Harrison, A (N.D) A Language Testing Handbook Modern English Publications
14 Henning, G (1987) A guide to Language testing Cambridge: Newbury House Publisher
15 Hughes, A (1988) Writing English Tess (2 nd Edition) London: Longman
16 Hughes, A (1989) Testing for Language Teachers Cambridge: Cambridge
17 Ibe, M D (1981) Language test analysis beyond the validity and reliability criteria (In) Papers on Language Testing Occasional Papers Read J.A.S (Ed)
No 18.pp1 Singapore: SEAMEO Regional Language Center
18 Kelly, R (1978) On the construct validity of comprehensive tests: an exercise in applied linguistics University of Queensland PhD thesis
19 Kerlinger, F N (1973) Foundations of behavioral research New York: Holt,
20 Moller, A.D (1982) A study in the validation of proficiency tests of English as a Foreign Language Unpublished PhD thesis, University of Edinburgh
21 Moore, G E (1992) The significance of research in vocational education: 1992
AVERA presidential address The Journal of Vocational Education Research, 17(4),
22 Oller, J.W Jr (1979) Language tests at school London: Longman
23 Read, J.A.S (1983) What is a good classroom test? (In) Guidelines A Periodical for Classroom Language Teachers Classroom tests Crabbe, D (Ed.) Volume 5
No.1.pp1-7.Singapore: SEAMEO Regional Language Center
24 http://www.cis.doshisha.ac.jp/kkitao/library/article/test/design2.htm
Appendix 1 The content of the course book
Contents Language Input Skills development
Using a bilingual dictionary Plural nouns
The family Opposite adjectives Food and drink
In a cafe An email from England
Verbs help, make, serve Jobs
Seumas Mc sopporran – the man with 13 jobs
Asking about a friend and relatives
Verbs: relax, go out, play Leisure activities
Leisure activities What is your favorite season?
A questionnaire – Do you have a healthy lifestyle?
My favorite season Where are they?
What is in your picture?
What is in your bag?
Countries and language Verbs Words that sound the same
Formal letters Applying for a job
Verb Words that go together
The year I was born What did it happen?
Relationships Spelling and silent letters Phonetic symbols
How did you two meet?
Three inventions How did you two meet
Food you like Role play- shopping Meals in your country
City and country adjectives City and country words
Directions 2 Viva la danza- Havana/
I have got more than you
A walk in the country/city
Comparing life in the city and country
Clothes Describing people Words that rhyme
Describing a person/ scene Getting information- who is at the party?
Who is at the party?
Born free Dangerous sports Interviews World weather
Present perfect and past simple
We’ ve never learnt to drive
Cities you have been to Things have been done
This questionnaire aims to enhance the quality of the final achievement test for non-major students Your honest and accurate responses are crucial for this improvement.
1 Does the test measure what you have been taught? a Yes b No
2 In your opinion, time allowance for this test is… a Not enough b Enough c Too much d Your own ideas
3 What do you think about the difficulty level of each part in the test? a very difficult b difficult c average d easy
4 Are all types of the test familiar with you? a Yes b No
5 Have you been taught grammatical structures and vocabulary in part 1,2 and 3 of the test? a Yes b No
6 The topic of the writing part is a Appropriate and relevant b Not related to what you have been taught
7 Which mark did you get for the final achievement test?
1 In your opinion, what is the purpose of the test? a To assess teachers’ teaching effectiveness c To classify or rank the students b To measure the students’ ability d To evaluate the effectiveness of a book
2 What did the teacher base on when designing the final achievement test for second year non major students? a Level of the students b The teaching and learning syllabus c The teacher’ book
3 Does the test measure what students have been taught? a Yes b No
4 Are all types of the test similar to what the students have been taught? a Yes b No
5 Have all the grammatical structures and words in the test been taught? a Yes b No
(If not, what grammatical structures and words aren’t taught?………
6 Is the topic of the writing part relevant to what students have been taught? a Yes b No
7 Please evaluate the difficulty level of each part in the test to the level of students
8 In your opinion, time allowance for this test is… a Not enough b Enough c Too much
(If too much or not enough, how much time do you think is appropriate for the test?
9 Before giving the test to students, was it discussed among teachers to find out its content validity? a Yes b No
Appendix 4 Answer key for reading task
1 Because he played cards for hours and got very hungry
2 in the oven of a stove
5 The cook cut some very thin pieces of potatoes and fried in oil Then he put salt on them
II Decide the following sentences true (T) or false (F)
Appendix 5 Answer key for new final achievement test
C: Our house is quite old, bout fifty years old It is in the centre of the village near the church All the houses here are white We have a living room, quite a big kitchen, and three bedrooms, and a big verandah all around the house
B: Our children are not at home now They both have jobs in the city so most of the time it is just Candy and me
C: Yes, so in summer we do bed and breakfast for tourists We have lovely visitors from all over the world
When I grow up, I am going to be a footballer- a really good one I am in the school team and I play three times a week But I am going to train very hard, everyday so I can be really, really good First, I am going to play for Manchester United, then Inter Milan, and then Real Madrid Those are my favorite teams I am going to travel all over the world and I am going to be famous I am not going to marry until I am very old- about
25 Then I want to have two sons I am going to play football until I am 35- that is very long time And I am going to teach my sons to play I want them to be famous footballers, too