INTRODUCTION
Rationale
The field of material development and evaluation has always been an important one in the history of English language teaching According to Cunningsworth (1995) and Ellis
Materials evaluation is crucial for English language teachers, as it enables them to move beyond subjective assessments and gain contextual insights into teaching resources According to William (1983), teachers must identify various criteria, models, and principles of materials evaluation to ensure that the materials are suitable for the teaching and learning process, while also recognizing their strengths and weaknesses This capability is essential for both educators and program designers, highlighting the importance of effective materials evaluation in enhancing language instruction.
At Hanoi University of Industry (HaUI), enhancing the quality of English language training is a key objective aimed at preparing students for suitable employment and developing their professional skills In 2017, the Faculty of Foreign Language (FFL) introduced blended learning materials for English for Occupational Purposes (EOP), which have since been widely implemented in specialized English courses during the initial years of university This blended learning model, combining online and face-to-face instruction, offers flexibility and affordability, fostering student autonomy and independence in language learning Consequently, students emerge more confident and aware of how to effectively use English in real-world scenarios Overall, blended learning for EOP is crucial in modern education, as it facilitates personalized learning experiences and enhances connectivity in the classroom.
Identifying effective EOP materials for local learning programs poses a challenge A key consideration is the adequacy of the English coursebooks developed at HaUI, particularly "English for Mechanical Engineering 3," in meeting the needs of both educators and students.
Besides, there are some issues that arise during the teaching process Although most
HaUI's English coursebooks focus on relevant topics to prepare students for their future careers; however, many do not fully utilize these resources to enhance their skills All undergraduate students are required to complete online lectures at home covering essential skills such as reading, listening, writing, vocabulary, and grammar, making it challenging for students with weaker foundational knowledge to keep up during in-person classes Additionally, the repetition of similar games and activities in each lesson can lead to a decline in student interest.
Since its implementation in 2018 for over 500 students at HaUI each semester, the course book "English for Mechanical Engineering 3" has not undergone an official evaluation by researchers or lecturers to assess its alignment with course objectives and student levels Therefore, conducting a study to evaluate the "English for Mechanical Engineering 3" book is essential to identify effective solutions for the challenges faced by local students in learning English This study aims to address these concerns.
Aims and objectives of the study
This study aimed to evaluate the relevance of the coursebook “English for Mechanical Engineering 3” in terms of 5 criteria: Aims, Contents, Methodology,
Technology and Design The specific aims are:
to examine the students and teachers‘ evaluation of the coursebook
to offer suggestions or adaptations for improving the effectiveness of the coursebook.
Research questions
The study was designed to answer the two following questions:
What is the students’ evaluation of the coursebook “English for Mechanical Engineering 3?
What is the teachers’ evaluation of the coursebook “English for Mechanical Engineering 3?
Scope of the study
This research paper, part of an M.A minor thesis, evaluates the coursebook "English for Mechanical Engineering 3," utilized by second-year Mechanical Engineering students at Hanoi University of Industry The study involved both teachers and students as participants Data was collected through student questionnaires and interviews to conduct a post-use evaluation of the coursebook's effectiveness.
The evaluation of three teachers was conducted using a comprehensive approach that integrates various models from multiple researchers, focusing on key themes such as Aims, Contents, Methodology, Technology, and Design.
Significance of the study
This paper aims to assist English teachers by providing valuable feedback on course contents and curriculum, enabling them to identify both strengths and weaknesses in their coursebooks Consequently, this insight will guide teachers in designing more effective lesson plans and allocating time appropriately for each lesson.
Evaluating the strengths and weaknesses of a coursebook is essential for enhancing English teaching methods By adapting and refining these methods, teachers can significantly improve the quality of their instruction, leading to increased student engagement and better outcomes in language learning.
This article provides valuable insights for teachers and designers in evaluating educational materials, serving as a reference for researchers focused on selecting appropriate resources that align with course objectives and cater to students' levels and interests.
Method of the study
There are two main instruments applied in the study to collect the data:
+ Survey questionnaires – were designed for second-year students majoring in
+ Semi- structured interviews – were used for asking teachers‘ views about the coursebook they are currently using and their suggestions to make it more efficient in the next few years.
Organization of the thesis
The study consists of five chapters presented as below:
Chapter 1: Introduction – provides a general overview of the study with specific references to the rationale, aim and objectives of the study, research questions, scope of the study, significance of the study and organization of the thesis
Chapter 2: Literature Review – provides a theoretical basis for the study with literature review on the aspects relating to materials in language teaching and learning; material
4 evaluation, blended learning for EOP materials evaluation and previous studies on materials evaluation
Chapter 3: Research Methodology – describes the methods used in this thesis It includes four main things: the setting of the study, the participants, data collection instruments and procedures and data analysis
Chapter 4: Research Result and Discussions – is devoted to a detailed description of document analysis and survey‘s results In this thesis, the main findings of the thesis are explored to address the research questions
Chapter 5: Conclusion – summarizes the content of the thesis with regard to the relevance of the coursebook The researcher proposed some adaptation techniques to make the coursebook more effectively Moreover, this study also includes the limitations of the study and gives suggestions for further research
LITERATURE REVIEW
Materials in language teaching and learning
Materials in language education are increasingly defined as systematic descriptions of techniques and exercises used in classroom teaching (Brown, 1995) They encompass various resources utilized by teachers and learners to facilitate language acquisition, including linguistic, visual, auditory, and kinesthetic elements, presented in formats such as print, live performance, or digital media (Tomlinson, 1998) Language materials, which include course books, videos, graded readers, flash cards, games, and websites, play a crucial role in enhancing the learning experience (Tomlinson, 2012) While early English Language Teaching (ELT) materials primarily consisted of reading texts with comprehension questions and grammar exercises, modern materials now offer comprehensive "packages" that include teachers' guides, workbooks, and multimedia support, promoting collaboration between teachers and learners to optimize lesson effectiveness (Littlejohn, 1998).
According to McGrath (2002), materials in education encompass a wide range of items, including tangible objects like pens and rulers, as well as visual presentations such as drawings and photographs He notes that textual materials utilized for language teaching and learning can include worksheets, computer software, course books, textbooks, and materials created by teachers.
Materials serve as essential learning resources, including teachers' guides, workbooks, coursebooks, textbooks, videos, and CD-ROMs, designed to enhance the teaching and learning process They aim to help achieve lesson objectives while also saving valuable time for educators Ideally, these materials should focus on facilitating learning rather than merely supporting teaching (Tomlinson).
2012, p 143) This definition of materials will be applied in this study
Materials play a crucial role in English Language Teaching (ELT), serving as an embodiment of the aims, values, and methods relevant to the teaching-learning context (Hutchinson & Waters, 1987) The selection of appropriate materials is one of the most significant decisions language teachers face According to Ur (2006), materials fulfill seven essential roles: they provide a structured framework for teaching, act as a systematic syllabus, offer ready-made texts and tasks tailored to students' levels, supply cost-effective resources, guide inexperienced teachers, and promote learner autonomy Furthermore, Tomlinson (2012) emphasizes that materials serve four main functions: instructional, experiential, eliciting, and exploratory, enabling learners to understand language targets, experience language use, and engage in active learning.
7 function); encourage them from using the language (eliciting function); and support them to discover about the language (exploratory function) That is to say, both Tomlinson
In their studies, Ur (2006) and (2012) highlight the essential role of materials in various writing styles, emphasizing their significance in English Language Teaching (ELT) Both authors agree that these materials are not only helpful but also necessary for effective teaching and learning in the ELT context.
While the benefits of using materials in education are widely acknowledged, various researchers also highlight potential drawbacks A significant concern is that an over-reliance on these materials may negatively impact learners' needs, as noted by Allwright.
Heavy reliance on teaching materials can diminish the impact of a teacher's contributions in the learning process According to Cunningsworth (1995) and other educational theorists like Tomlinson (2003), Ur (1996), and Williams (1983), no single material can meet the diverse learning styles and needs of every classroom Teachers who depend too much on these resources without thoughtfully designing their lessons risk causing student boredom and discouragement in English classes.
In the context of Vietnamese universities, particularly at Hanoi University of Industry, teaching materials play a crucial role in enhancing the educational experience by aiding teachers in lesson design and promoting student autonomy in learning However, it is essential for both educators and students to avoid over-reliance on these materials To improve lesson effectiveness, a variety of supplementary resources should be integrated, allowing for a more comprehensive approach to education Additionally, teachers must demonstrate flexibility in adapting materials to suit the diverse levels and abilities of their students, ensuring a more tailored and impactful learning environment.
Choosing the right materials for learners is a critical concern for administrators, teachers, and students alike, often leading to a dilemma between published and teacher-produced resources Regardless of the choice made, the primary objective remains to enhance the learning process This section will explore the advantages and disadvantages of both published materials and teacher-produced materials.
Commercial materials refer to standard educational resources produced for the market, tailored to meet classroom teaching needs Typically developed by book publishers, these materials align with current curriculum standards and are designed to support specific courses through targeted exercises and content.
Successful course books are characterized by their usefulness and ease of use for teachers Their effectiveness often stems from their clever design and thoughtful construction, making them highly practical for classroom settings.
Less successful course books often exhibit inherent flaws in their construction, resulting from hasty assembly or inadequate editing and field-testing Consequently, these materials are frequently viewed as challenging to use and unengaging (Richards, 2004).
Educational materials offer both benefits and drawbacks While they assist teachers in discovering effective teaching methods, their effectiveness can diminish if used inappropriately or in unsuitable contexts Additionally, if materials fail to engage both teachers and learners, they may be perceived as challenging and uninteresting Cunningsworth highlights these disadvantages, emphasizing the need for careful selection and application of teaching resources.
In 1995, it was highlighted that the abundance of available published materials poses a significant challenge for teachers and designers in selecting appropriate textbooks and supporting resources This sentiment aligns with Richards' observations in 2014 regarding the difficulty of making informed choices amidst the popularity of various educational materials.
Richards (2004) discussed the pros and cons of commercial materials, while Block (1991) advocated for the use of teacher-produced materials, highlighting their advantages and disadvantages Block argued that despite some drawbacks, teacher-produced materials can be more beneficial than commercial options, serving as effective resources to support student learning One key advantage of homemade materials is their contextualization; they are tailored to meet the specific needs of a particular group of learners or educational setting Additionally, these materials cater to individual needs, reinforcing Block's assertion that they can enhance the learning experience.
Materials evaluation
2.2.1 Definitions and purposes of materials evaluation
Many researchers have expressed their ideas about the field materials evaluation Firstly, to define ―evaluation‖, Dudley-Evans and St John (1998:128) states that
Evaluation is a comprehensive process that starts with identifying the necessary information and culminates in implementing changes to current practices or shaping future ones According to Hedge (2000), while evaluation traditionally referred to assessing students at the end of a course, its scope has expanded to encompass all facets of a program, including teaching materials Specifically, materials evaluation is a systematic procedure that assesses the value or potential of learning materials, as described by Tomlinson (2003) This principled approach provides practical insights into the targeted materials, ensuring they are selected or developed in a reliable and valid manner.
Materials evaluation is a crucial aspect of curriculum development, involving a systematic process of data collection and analysis This process assesses educational materials based on their alignment with specific aims, objectives, and the needs of learners Furthermore, it is essential that evaluation translates into actionable insights for effective implementation.
Materials evaluation is essential for determining their suitability for specific educational purposes, as highlighted by Hutchinson & Waters (1987), who state that evaluation involves assessing the fitness of materials rather than categorizing them as simply good or bad This process is crucial to ensure that the selected materials effectively support students' learning objectives and needs.
Sheldon (1988) emphasizes the importance of materials evaluation in education, highlighting its role in helping university and college staff differentiate between various available resources This process not only provides teachers with valuable insights into the strengths and weaknesses of the textbooks they use but also enables them to leverage the advantages of these materials while addressing any shortcomings in specific tasks or activities.
Rea-Dickins and Germaine (1992) pointed out some purposes of materials evaluation They are divided into two types: general purposes and specific, topic-related purposes
For general purposes of evaluation, they claim three main reasons as follows:
Self-development: teachers and other language teaching professionals
The primary objectives of evaluation for accountability focus on assessing coursebooks to determine their quality and value for money Additionally, curriculum development evaluations gather insights from teachers and staff, while self-development evaluations for teachers emphasize understanding their unique contexts to enhance their teaching practices.
Evaluation serves as a crucial tool for assessing efficiency, allowing us to identify what is effective, what is less effective, and what is ineffective in specific, topic-related contexts.
Tomlinson and Masuhara (2004) emphasize that the primary goal of materials evaluation is to facilitate the redevelopment of educational materials They highlight the importance of identifying both the strengths and weaknesses of these materials to inform effective selection and development processes.
The primary goal of materials evaluation is to assess the strengths and weaknesses of educational resources to select the most suitable materials for teaching and learning In this study, the focus is on evaluating the effectiveness of course books in relation to their aims, objectives, and the specific levels and interests of students Ultimately, the study aims to determine the relevance of the coursebook by examining its aims, content, methodology, technology, and design.
Each researcher has different views about types of materials evaluation
Ellis (1997) identifies two key types of materials evaluation in education: predictive and retrospective Predictive evaluation is concerned with selecting appropriate materials before their use, while retrospective evaluation assesses the effectiveness of materials that have already been implemented.
According to Cunningsworth (1995:76), McGrath (2002:14-15) and Tomlinson
(2003), there are three types of materials evaluation: pre-use, in-use and post-use Compared to Ellis‘s material evaluation; material evaluation conducted by these authors
13 is clearer and more detailed as post-use evaluation is applied after the students have just already finished the coursebook
Pre-use evaluation is essential for assessing textbooks to identify their strengths and weaknesses, ensuring the selected course book aligns with the course objectives and students' levels According to Tomlinson (2003), this process involves predicting the potential value of materials for users However, it is often subjective and unreliable, as teachers may only skim through a book for a quick impression of its worth.
In-use evaluation is a process that assesses the strengths and weaknesses of a course book currently utilized in a classroom setting By monitoring students' success in achieving course objectives and their overall learning progress, teachers and designers can make informed decisions about the continued use of the material According to Tomlinson (2003), this evaluation method relies on observation and short-term outcomes, making it more objective and reliable compared to pre-use evaluation.
Post-use evaluation, often overlooked in educational settings, is crucial for enhancing teaching and learning processes Also known as end-of-program evaluation, it assesses the effectiveness of a coursebook after its implementation According to McGrath (2002), this evaluation helps determine whether a coursebook should be changed, adapted, supplemented, or retained as is Tomlinson (1998, 2003) emphasizes that post-use evaluation is the most significant form of materials evaluation, as it measures the actual impact on users and provides reliable insights This evaluation can also capture short-term effects, including motivation, impact achievability, and immediate learning outcomes.
Robinson (1991:59) identifies three types of materials evaluation in English for Specific Purposes (ESP): preliminary, summative, and formative Preliminary evaluation occurs prior to the start of an ESP course, focusing on the selection of suitable course books In contrast, summative evaluation is conducted at the course's conclusion to assess the effectiveness of the course materials for both teachers and learners Formative evaluation takes place during the course to gather insights and improve the learning experience.
Robinson (1991) emphasizes the ease of modifying materials and asserts that evaluation can be conducted by both insiders and outsiders He differentiates between process evaluation, which focuses on teaching strategies and decision-making, and product evaluation, which assesses student outputs like essays and exam results Key evaluation techniques identified include tests, questionnaires, checklists, rating scales, interviews, and observational records.
Materials evaluation in research, while varied in terminology, shares fundamental similarities across different scholars For example, Robinson's preliminary evaluation (1991) aligns closely with Cunningsworth's pre-evaluation (1995), as well as the approaches of McGrath (2002) and Tomlinson (2003) Each type of evaluation has distinct advantages and disadvantages, making it crucial for researchers to select the most appropriate method for their specific context In the case of Hanoi University of Industry, materials are categorized into three types: pre-use evaluation, while-use evaluation, and post-use evaluation This study specifically emphasizes post-use evaluation to assess the effectiveness of course books in relation to course objectives and the needs, levels, and interests of students.
There are some different models of materials evaluation developed by some famous researchers and the popular frameworks are suggested by McDonough and Shaw (1993), Hutchinson and Waters (1987)
2.2.3.1 McDonough and Saw’s model of the process for material evaluation
Blended learning for English for Occupational Purposes (EOP)
Blended learning emerged in the late 1990s as an innovative teaching approach that leverages technology and the internet to promote student autonomy and encourage educators to adopt more effective teaching methods This shift transitions from a teacher-centered model to a student-centered model, as highlighted by Talor (1995) While there has been ongoing discussion regarding the definition of blended learning, its terminology became more clearly defined around the early 21st century, as noted by Laster, Otte, and Picciano.
Blended learning, as defined by 19 and Sorg (2005) at the Sloan-C Workshop, combines online activities with traditional face-to-face class interactions in a structured manner, replacing some in-person time with digital engagement Graham (2006) further elaborates that this approach merges face-to-face instruction with computer-mediated learning Currently, "blended learning" refers to the integration of digital education with conventional classroom settings, necessitating the physical presence of both teachers and students (Friesen).
Blended learning combines online and face-to-face instruction, with 30-80% of course content delivered online This educational approach allows learners to engage in formal programs while gaining some control over their learning experience, including aspects such as time, place, path, and pace, thereby enhancing the overall learning process (Poon, 2013).
Blended learning can be categorized into enabling blends, enhancing blends and transforming blends in terms of a learning scope (Graham, 2006)
Enabling blends provide learners with convenient and accessible learning opportunities, enhancing flexibility through online tools and resources For example, students can access audio or video recordings of in-person lectures, allowing them to catch up on missed classes or review lesson content While the core pedagogy of a course remains largely unchanged, enabling blends serve as a valuable additional option for learners (Lindquist, 2012).
Enhancing blends significantly transform course pedagogy by integrating online materials into face-to-face classes For example, in a blended learning English course, traditional in-class lectures are minimized and replaced with interactive lessons This shift allows for more collaborative activities, such as pair and group work, enabling students to engage more actively in their learning process This approach is widely adopted in universities and colleges (Graham, 2012).
Transforming blends facilitate a significant shift in pedagogy, transitioning from a passive learning model to one where students actively engage in constructing their own skills and knowledge through interactions with peers in both online and in-person environments This necessitates a substantial redesign of face-to-face teaching and learning practices to effectively support and enhance online learning experiences.
To put it simply, this type of learning is identified as ―thoughtful fusion‖ of face-to-face and online learning (Garrison& Vaughan, 2008, p.5)
Blended learning offers several significant benefits for both teaching and learning It ensures continuity in education during school closures (Riel and Paul, 2009) and fosters active and flexible learning, enabling students to express their needs and interests for greater success (Pape, 2012) This approach enhances access to knowledge, promotes student autonomy, and cultivates independent language learners who are more confident and aware of their language use in real-world situations Additionally, blended learning provides teachers with insights into individual learning experiences, allowing them to tailor their support and monitor progress effectively (Joosten et al., 2013) The use of online learning records and assessment tools equips educators with valuable information about what students know, how they acquire knowledge, and their emotional engagement with the learning process (Murphy, 2008).
Blended learning combines traditional classroom instruction with online learning, allowing for a more enriched educational experience This approach enables learners to engage more deeply while giving teachers the ability to maintain control over various learning modes and schedules.
Since its implementation in 2018 at Hanoi University of Industry, the blended learning model has significantly transformed teaching by emphasizing student-centered learning over traditional teacher-centered approaches This model encourages students to complete online exercises prior to attending class, fostering a more individualized and connected learning experience A comprehensive discussion of this model will be provided in Chapter 3: Methodology.
2.3.2 English for Occupational Purposes (EOP)
English for Occupational Purposes (EOP) is a specialized area within English for Specific Purposes (ESP) designed to address the English language needs of learners in their professional environments EOP courses are beneficial in both academic settings, where students prepare for future careers, and in business contexts For instance, a technician handling customer complaints about technical issues may require specific language skills to effectively communicate and resolve problems.
- Great the customer and ask for his name
- Ask for the details of the problem
- Ask for details of the product (model, order number…)
EOP courses, including English for Mechanical Engineering, Nursing, Business, and Accountants, are designed to address specific communication needs in various professional fields As job-related communication becomes more complex, these courses focus on detailed and specialized language skills rather than attempting to cover a broad spectrum of workplace requirements.
A key aspect of English for Occupational Purposes (EOP) is its emphasis on learner-centered strategies, which prioritize the needs and interests of students in curriculum development To create an effective EOP curriculum, it is crucial to conduct a thorough needs analysis rather than relying solely on perceptions or intuitions.
According to Kim (2008, 76) suggests some of the following emerging features of EOP in recent times when he collected data from employees‘ interviews at Korean companies
- It has clear purpose (e.g., business skills for presentations in English, business letter writing, etc.);
- It deals with needs in proficiency as well as a need‘s analysis or needs in workplace;
- The relevant vocabulary and expressions are link to their workplace;
- It is more appropriate to immediate needs, and assists more practical purposes than general English in the workplace
HaUI offers a variety of English for Specific Purposes (EOP) courses, including English for Mechanical Engineering and English for Electrical and Electronic Engineering, designed to cater to the diverse needs and interests of learners These specialized courses enhance students' career prospects post-graduation by providing detailed and targeted content that aligns with their professional goals A comprehensive discussion of EOP will be presented in Chapter 3: Methodology of this research.
2.3.3 Blended learning for EOP materials
Blended learning in English Language Teaching (ELT) integrates computer-assisted language learning (CALL) with traditional face-to-face instruction, allowing teachers to access diverse online resources tailored to their specific teaching needs These resources include videos, texts, audiotapes, narrated slide shows, and online quizzes, often accompanied by engaging activities that can be completed individually or collaboratively The incorporation of Information and Communication Technology (ICT) not only enhances learning materials but also broadens interaction among students and between students and teachers, facilitating both in-person and online communication.
According to Mayer (2005) cited in Gruba and Hunelman, 2002a, he proposed a framework for blended learning
Figure 2: A framework for blended language learning by Mayer (2005 )
The materials dimension focuses on how educational materials are presented to students, emphasizing their learning impact rather than their generation or delivery This perspective categorizes texts along a continuum from static to dynamic, with printed handouts, images, web pages, and textbooks considered static In contrast, dynamic materials include PowerPoint presentations with Prezi, recorded audio-visual content like YouTube and Skypecasts, podcasts, and live texts produced interactively through SMS messaging and online chat This leads to the learner-material dimension, highlighting the two-way interaction between learners and materials.
‗symbiotic‘ relationship‖ This is because materials at the ‗dynamic‘ end of the spectrum in particular, are learner-produced and learner-influenced‖ (Mayer, 2005 cited in Gruba and Hunelman, 2002a)
Blended learning for EOP materials evaluation
Blended learning for English for Occupational Purposes (EOP) materials is evaluated based on several criteria recommended by researchers, primarily focusing on the work of Oliver and Herrington (2001) The first key criterion is pedagogies, emphasizing that quality online units should prioritize activity-based learning over content delivery, allowing students to engage in complex, real-life tasks collaboratively, with teachers acting as coaches rather than traditional instructors The second criterion, resources, highlights the importance of providing diverse content that presents multiple perspectives, enabling students to critically assess different viewpoints rather than relying solely on the teacher's opinion Lastly, delivery strategies pertain to the methods used to convey course material to learners, ensuring effective transfer of knowledge These three criteria are essential for developing robust online programs at HaUI.
Nautaa, J.M., Platenkamp, J., and Hettingaa, M (2016) proposed three essential criteria for evaluating blended learning materials Firstly, they emphasize the importance of diverse study materials, including instructional videos and regularly updated online assignments Secondly, they highlight the need for user-friendly digital resources, ensuring that materials are easy to access and download Lastly, clarity in the content is crucial, as it enables learners to effectively grasp the knowledge being presented These interconnected criteria collectively enhance the quality of blended learning experiences.
Moreover, this research also applies criteria suggested by Bowyer& Chambers (2017) related to aims, contents (online & in class) and technology for evaluating blended learning materials
Criteria Evaluating blended learning materials
1 Aim Some elements of aim need to evaluate such as the scope or learning objectives
2 Content Some elements of content need to evaluate such as: Quality (content and present gain attention); Validity (Content is integral); Accuracy and balance (Vocabulary and terminology used are appropriate for the learners); Interactivity (there is interactivity in lecture notes) Accessibility (Given examples are up-to-date); Organization (Exam duration are well-organized); Currently (The content is up-to-date); Support (Lecture notes are supported by multimedia tools: video, audio, etc.)
3 Technology Some elements of technology need to evaluate such as: Interface design, Ease of use, Reliability, Usability, Availability, Interactivity, Currency (up-to-dateness); Support
Table 1: Criteria to evaluate blended learning materials suggested by Bowyer, J (2017)
Nezhad, H.Y., Atarodi, I., and Khalili, M (2013) introduced four additional criteria for evaluating and comparing two English for Specific Purposes (ESP) coursebooks focused on Mechanical Engineering These criteria include vocabulary and grammar, exercises and activities, text attractiveness, and physical presentation, as well as contextual relevance.
Criteria Questions to evaluate ESP coursebooks
1 Content 1.1 Is the subject matter presented either topically or functionally in a logical, organized manner?
1.2 Are the reading selections authentic pieces of language?
2.1 Are the grammar rules presented in a logical manner and in increasing order of difficulty?
2.2 Are the new vocabulary words presented in a variety of ways (e.g., glosses, multi glosses, and appositives)?
2.3 Are the new vocabulary words repeated in subsequent lessons to reinforce their meaning and use?
3.1 Are there interactive and task-based activities that require students to use new vocabulary to communicate?
3.2 Do instructions in the textbook tell students to read for comprehension?
3.3 Do the activities facilitate students‘ use of grammar rules by creating situations in which these rules are needed?
3.4 Do the exercises promote critical thinking of the text?
4 Attractiveness of the Text and
4.1 Is the cover of the book appealing?
4.2 Is the visual imagery of high aesthetic quality?
5 Context 5.1 Does the text coincide with the course goals?
5.2 Is the textbook appropriate for the students who will be using it?
5.3 Is the textbook appropriate for the teacher who will be teaching it?
Table 2: Criteria to evaluate ESP coursebook suggested by Nezhad, H.Y., Atarodi, I., & Khalili, M (2013)
Ali Isik (2018) proposed essential criteria for evaluating materials in English for Specific Purposes (ESP) courses, emphasizing the importance of content criteria in assessing ESP coursebooks Unlike previous researchers, Isik introduced a comprehensive set of criteria that not only applies to English as a Second Language (ESL) but also to ESP materials He highlighted the need for researchers to examine whether ESP courses address specific language skills, content, texts, tasks, terminology, symbols, and acronyms, as well as the inclusion of real-life simulations and workplace task demands Additionally, he stressed the significance of unit formats, including warm-up, body, and follow-up activities, and the evaluation of supporting software to determine the usability of ESP coursebooks Isik's detailed and practical content and software criteria will primarily guide this study.
The author develops a unique framework for evaluating the coursebook "English for Mechanical Engineering," drawing from various criteria proposed by researchers such as Hutchinson & Water (1987), Oliver & Herrington (2001), Tomlinson (2003), Dickinson (2010), Bowyer (2017), Nezhad et al (2013), and Işik (2018) This framework consists of five key criteria: Aims, Contents, Technology, Methodology, and Design.
3‖ at HaUI Here is the framework adapted from different authors and what can be applied in this study can be seen in the Appendix 5
Figure 5: Researcher’s own framework adapted from different authors
Previous studies on materials evaluation
Research has been conducted to assess blended learning for English for Occupational Purposes (EOP) and English for Specific Purposes (ESP) materials, highlighting various effective approaches to materials evaluation This article will discuss several related studies focused on evaluating educational materials in this context.
In 2010, Shih, R C conducted a study on blended learning through video-based blogs for an English for Specific Purposes (ESP) course titled English Public Speaking The research framework identified four key elements of blended learning: instructor, technology, student, and classroom, as highlighted in the literature review.
Figure 5: A blended learning model for an English Public Speaking course by Shih, R C (2010)
The research methodology integrated qualitative and quantitative approaches, incorporating peer and instructor feedback, interviews, self-reflection, and a learning satisfaction survey with 44 final-year Business English students The findings indicated that the blended learning model effectively enhanced student learning and satisfaction, provided it is supported by adequate resources and comprehensive lesson plans Key factors contributing to learner satisfaction included the feedback from peers and instructors, as well as the blog's features such as free access, ease of revision, and engaging learning materials.
Klimova (2017) conducted research on the blended learning model and its materials in the Business English course at the Faculty of Informatics and Management in Hradec Kralove, Czech Republic, aiming to assess its efficiency and learning outcomes in an international context The study utilized a case study approach, questionnaire surveys, and online course reports Findings indicated that the blended learning model was not particularly effective in achieving desired educational results.
BL approach in teaching Business English although the learners were interested in BL strategy and preferred it to traditional learning approach
A study by Kalugina et al (2018) evaluated the effectiveness of English for Specific Purposes (ESP) teaching in a blended learning environment utilizing a smart coursebook Involving 48 undergraduate students from a non-linguistic university, the research compared pre-test and post-test results to assess the impact of blended learning strategies on student performance Additionally, questionnaires were used to gather student feedback on their experiences with ESP blended learning The findings indicated that participants found blended learning practical and beneficial for enhancing their English competence, as evidenced by improved post-test scores.
Blended learning for English for Specific Purposes (ESP) and English for Occupational Purposes (EOP) has shown limited effectiveness, particularly in studies conducted by Klimova (2017) and Kalugina et al (2018) Consequently, the researcher has opted to implement blended learning specifically for EOP materials.
The evaluation of "English for Mechanical Engineering 3" aims to determine its effectiveness in meeting the learning outcomes aligned with occupational purposes, as well as addressing the needs and interests of learners at HaUI.
Chapter summary
This chapter has reviewed the literature that underpins the theoretical framework for the research titled "An Evaluation of the Coursebook 'English for Mechanical Engineering 3' at a Technical University in Hanoi." It critically discusses various aspects, including definitions of materials in language teaching and learning, materials evaluation, and the role of blended learning in English for Occupational Purposes (EOP) and its materials Ultimately, effective evaluation requires a systematic approach and a comprehensive, flexible framework.
METHODOLOGY
An overview of the research contexts and the evaluated coursebooks
The study was conducted at Hanoi University of Industry, where the coursebook
“English for Mechanical Engineering 3” compiled by a group of English teachers at
Established in 2018, the Faculty of Foreign Language offers six specialized modules in English for Mechanical Engineering, ranging from Module 1 to Module 6 These modules are designed to enhance the language skills of students in the field of mechanical engineering, ensuring they are well-equipped for professional communication.
2, the contents of course focus on General English for daily communication; however, they pay more attention to English for Specific Purposes (ESP) from module 3 to 6
Module 3 comprises 13 classes, each averaging 34 students HaUI's classroom models increasingly embrace blended learning, integrating face-to-face instruction with the eop.edu.vn courseware This supportive learning platform is designed to enhance English knowledge and skills for students majoring in Mechanical Engineering.
The English for Occupational Purposes (EOP) program offers six courses from the first to third year, each consisting of 75 periods, which includes 40 offline and 35 online sessions Students engage in online self-study to enhance their English skills in listening, reading, writing, vocabulary, and grammar, while face-to-face classes focus solely on speaking skills Over a span of 10 weeks, students attend four classes per week, and to complete each module, they must attend classes and pass three tests during the course, along with two final assessments: a written test covering various skills and a spoken test that evaluates personal information, pair conversation, and free talk.
Developing English skills is central to the university curriculum, aiming to enhance students' career prospects and future development.
The coursebook "English for Mechanical Engineering 3," authored by nine educators from the Faculty of Foreign Languages in 2018, is a topic-based resource designed for mechanical engineering students It comprises eight units, each focusing on distinct topics relevant to the field.
describe an unforgettable experience (Unit 1)
make phone calls; make and change arrangements (Unit 2)
describe features of objects (materials, properties, measurements); ask and answer about measurements of items (Unit 3)
describe a tool; describe a toolbox (Unit 4)
place and change orders (Unit 5)
complain and deal with complaints (Unit 7)
talk about safety rules in a workshop (Unit 8)
Besides, based on the syllabus in the coursebook, there is one revision which lasts 45 minutes after every 3 units and one wrap-up lesson at the end of the course
A typical lesson includes some of the following parts:
Warm-up activities, typically lasting 3-5 minutes, are designed to capture students' attention, activate prior knowledge, and help teachers introduce new topics engagingly Various interactive warm-up activities featured in the coursebook include whispering, matching games, hangman, and lucky number, all aimed at enhancing the learning experience.
Activity 1: With the combination of different games, students have a chance to review vocabulary so as to remember and know how to write or pronounce new words correctly
In this activity, students engage in a fun and interactive experience designed to enhance their sentence recall, sentence construction, and conversational skills, which ultimately prepares them for cue-card exercises and role-playing activities.
In Activity 3, known as the Speaking Card, students collaborate in pairs to engage in conversations that reflect real-life situations Each student alternates asking and answering questions to complete the dialogues, ensuring they construct grammatically correct sentences and incorporate useful language expressions for meaningful interactions.
Activity 4 (Role-play): Students talk about the topics related to English for
Mechanical Engineering education focuses on suggested questions that enhance students' self-confidence and communication skills with peers, ultimately aiding in achieving occupational learning outcomes The coursebook is specifically designed to support these objectives.
provide students with some basic knowledge (vocabulary, grammar structures, useful language expressions) and language skills related to different topics in the field of Mechanical Engineering
provide students with opportunities to practice and develop their language skills (Listening, Reading, Writing, especially Speaking) in general contexts as well as Mechanical Engineering ones
After the course, students will be able to:
In the field of Mechanical Engineering, it is essential to utilize foundational knowledge and language skills to effectively communicate on various specific topics This includes sharing unforgettable experiences, making and modifying phone arrangements, describing the features of tools and toolboxes, placing and altering orders, reporting technical issues, handling complaints, and discussing safety regulations in a workshop environment Mastery of these communication skills is crucial for successful interactions within the industry.
listen to short monologues or conversations (2.5-3 minutes) to identify short, specific pieces of information/ factual information (listening texts related to 8 topics in the coursebook)
read comprehension about the texts/ articles (250-350 words) related to 8 topics in the coursebook
write paragraphs/ emails about 8 learnt topics in the coursebook
practice conversation/ small talks about 8 learnt topics in the coursebook.
The participants of the study
This study focused on a target population of around 442 second-year Mechanical Engineering students and 8 English language instructors responsible for the "English for Mechanical Engineering 3" course at HaUI.
The researcher conducted a study involving 8 English teachers specializing in Mechanical Engineering at HaUI, representing the entire population of such educators Additionally, 200 non-English-major students, aged 18 to 22, were randomly selected from 6 different classes to participate in interviews and complete questionnaires.
In a survey involving eight English teachers from the Faculty of Foreign Languages specializing in Mechanical Engineering, four held M.A degrees and four had B.A degrees in English The teachers, aged between 27 and 40, included one male educator Each teacher boasted over six years of experience in teaching General English, but only two years of experience in English for Occupational Purposes (EOP) courses, which began with the introduction of the coursebook in 2018.
A total of 200 out of 442 Mechanical Engineering students participated in the research, representing six randomly selected classes at HaUI The students hailed from various provinces, including Nam Dinh, Nghe An, Ha Nam, and Ha Noi, with most having prior English study While students from Hanoi exhibited strong English proficiency, those from other provinces demonstrated intermediate levels Having recently completed eight units of the "English for Mechanical Engineering 3" coursebook, the students were tasked with evaluating the book based on their personal experiences.
Data collection instruments and procedures
Survey questionnaires were utilized in this study as a key instrument for data collection Litz (2005) highlights their effectiveness in gathering information from a large pool of respondents, making them practical for identifying objectives and assessing teaching and learning environments Additionally, Gillham (2000) notes that a significant advantage of survey questionnaires is that respondents can answer questions anonymously, which enhances the objectivity of the results.
In this study, a questionnaire was utilized to gather students' perspectives on the objectives, content, methodology, technology, and design of educational materials The survey comprised 43 questions, featuring multiple-choice options and one open-ended question, organized into six distinct sections.
Section 1: was designed with three questions so as to collect learners‘ opinions about the aim of the coursebook
Section 2: aimed at seeking for the information about the content of the coursebook which consists of 22 questions
Section 3: consisted of 5 questions with the aim of finding out students‘ evaluations on teaching methodology
Section 4: was designed to ask for students‘ opinions about technology which has 8 questions
Section 5: had 4 questions and was mentioned to get students‘ opinions about the design of the coursebook
Section 6: prepared one open-ended question to look for students‘ comments or suggestions to improve the effectiveness of the coursebook in the future
On May 4, 2020, questionnaires were distributed to learners and collected by May 11, 2020 To facilitate responses, the majority of questions were presented in multiple-choice format and translated into Vietnamese Students were instructed to indicate their agreement using a Likert scale (1=strongly disagree; 2=disagree; 3=agree; 4=strongly agree) for most key questions, with the exception of the final open-ended question.
In this study, semi-structured interviews served as a key data collection instrument, offering an open and flexible approach that enhances the depth of research findings This method not only supplements information but also ensures the reliability of responses, allowing participants to provide open-ended answers that yield more comprehensive insights.
In this research, eight Vietnamese teachers specializing in English for Mechanical Engineering were interviewed to address specific questions, along with some unplanned inquiries for deeper insights The interviews encouraged the teachers to share their perspectives and feedback, enriching the study's findings.
37 explanations, recommendations to evaluate the coursebook based on five criteria (Aim, Contents, Methodology, Technology, Design) Each interview was about 10- 20 minutes, which was recorded for summarizing the contents later
20 questions were asked in the interviews as following:
Section 1: was compiled to ask teachers‘ opinions about the aims of the coursebook
Section 2: was designed to look for teachers‘ opinions about the contents of the coursebook (including 5 questions)
Section 3: consisted of 5 questions related to methodology
Section 4: has 1 question to ask teachers‘ comments about technology
Section 5: aimed at asking teachers about the design of the coursebook (including 4 questions)
Section 6: Two questions were created to ask teachers‘ comments or suggestions to improve the effectiveness of the coursebook in the future
3.3.3 Design of the study and data collection process
The researcher gathered the data through survey questionnaires for students and semi-structured interviews for teachers
Data collection from the questionnaires
Step 1: Prepare beforehand and ask for consent: In order to finish a questionnaire, the researcher received a help from her supervisor to certify its contents Next, the author contacted Hanoi University of Industry to collect data directly The researcher also asked for the teachers‘ consent to partake in the survey
Step 2: Collect the data: The survey questionnaires were given online to all the second-year students majoring in Mechanical Engineering when they had just finished the course The researcher delivered to 6 random classes to answer the survey Each class has 34 students and the total surveys received was 200 students; however, 4 participants did not answer the surveys because they had just dropped out of classes Actually, collecting survey questionnaires was not easy as the researcher had to ask many times to convince students to do the surveys The researcher also asked a help from some teachers to persuade them to do In general, 200 surveys were used to serve the research‘s needs
Step 3: Evaluate the data collection process: After collecting the questionnaires from the students, the survey data were ready to be analyzed
Data collection from the interviews
Step 1: Contact: The researcher kept in touch with nine English teachers to confirm their agreements on semi-structured interviews After that, detailed information related to place and time of the interviews was provided
Step 2: Interview: To make clear all the data, the semi-structured interviews were conducted separately with eight English teachers to encourage them from giving their comments or recommendations to improve the effectiveness of the coursebook and then recorded to support data analysis There were not any difficulties when inviting the teachers to answer the questions as all of them were willing to participate so as to change and adjust the coursebook to make a better fit in the next coming years
Step 3: Collect data: After interviewing all the teachers, the researcher analyzed and transcribed the data interviews from recordings.
Data analysis
This study utilized two data collection instruments, a survey questionnaire and semi-structured interviews, employing both descriptive statistics and interpretative methods for data analysis The analysis of the questionnaire data followed three key steps.
Step 1: Examine the data: When receiving the questionnaires, the researcher examined thoroughly whether (1) the questionnaires were done; (2) the given answers were reasonable and understandable; (3) the given data responded to the questions Then, the researcher moved to the next step
Step 2 : Report the data: Data from 200 surveys were analyzed and synthesized based on the sections of the questionnaire In each section, the frequency of each option was calculated in percentages Results of the questionnaires helped answer the first research question about the students‘ evaluation of the coursebook ―English for Mechanical Engineering 3‖
Step 3: Visualize the results: Besides presenting in written form with descriptions and numbers, the result was illustrated in bar charts, pie charts or tables to generalize the data
Besides quantitative data from questionnaires, qualitative one from interviews were completed to answer the second research questions about the teachers‘ evaluation on the coursebook ―English for Mechanical Engineering 3‖
Step 1: Transcribe the recordings: In the first step, the researcher listened to the recordings of nine teachers interviewed before transcribing into written form for further analysis
Step 2: Choose the relevant data: Once all the recording had been transcribed, only suitable data to answer the second research question were chosen for in-depth analysis
Step 3: Conclude the findings: The findings were reported and discussion was concluded correspondingly.
Chapter summary
This chapter provides a comprehensive overview of the research context, the evaluated coursebook, and the study participants It clearly outlines the instruments and procedures used for data collection and analysis in the post-use evaluation The research employs document analysis, student survey questionnaires, and semi-structured teacher interviews, all adhering to the established criteria set forth by Hutchinson & Water (1987), Oliver & Herrington (2001), Tomlinson (2003), Dickinson (2010), Mukundan et al (2011), Bowyer (2017), and Nezhad et al (2013), as well as Ali Isik (2018) These criteria encompass key aspects such as Aim, Contents, Methodology, Technology, and Design.
