In order to keep up with the trend of globalization, the Ministry of Education (MOE) in Taiwan had to outline some proposals for educational reform, including several proposals for the area of English language instruction. Educational reform in the country started in 1997 and is still an ongoing project at present. According to the Nine-Year Integrated Curriculum Guidelines, the MOE initiated a new curriculum based on the Education Reform Action Plan for elementary and junior high schools in Taiwan in 1997 (MOE, Taiwan 2004). For years, English language instruction in Taiwan had been designed to begin in the first year of junior high school. However, with the Nine-Year Integrated Curriculum, English language instruction was advanced to fifth and sixth grade in 2001 and to third and fourth grade in 2005 (MOE, Taiwan 2004; MOE Taiwan 2006). The second change in educational reform was the opening of textbooks for elementary and junior high schools to non-governmental publication. Previously, all textbooks were designed and published by the National Institute for Compilation and Translation. At present, textbook policy in Taiwan has shifted from a unified editorial system to an open examination and appraisal system (Huang 2005).
The MOE of Taiwan (2004) has announced the newest curriculum guidelines for elementary and junior high school. In the English Language Study Area, English learning has been divided into two stages--elementary school and junior high school. The elementary school stage starts at third grade and ends at sixth grade. The junior high school stage, extending from the elementary school stage, starts at seventh grade and ends at ninth grade. There are three curriculum goals for elementary and junior high school students. The first one is to foster students' basic communication abilities and enable them to use the target language in real situations. The communication abilities include listening, speaking, reading, writing and integrated skills. The Nine-Year Integrated Curriculum Guidelines put much emphasis on listening and speaking in the first stage of the curriculum, i.e. the elementary school stage. In the junior high school stage, the balance of the development on the four skills is stressed. However, most of the time, the four skills would not be used in isolation, so the English Ability Indictor (the EAI) also takes the integration ability into account. The second goal of the EAI is to cultivate stu dents' interests and methods in learning English, so they can learn spontaneously and effectively. The last one is to enhance students' recognition of domestic and foreign culture so they can appreciate and respect the differences among cultures. The EAI lists what we expect students to achieve in the two stages of curriculum.
Though all the English textbooks on the market have been carefully reviewed and approved by the MOE, this study proposes to see how they fulfill the requirements of the curriculum guidelines and to offer some insights to school administrators and teachers when they have to choose English textbooks for their students. To these ends, the first purpose of this study is to examine the suitability of the design of language skills training in junior high school English textbooks approved by the MOE. The second goal is to find out if the textbook design fits students' needs. The third one is to identify the strong and weak qualities of textbook design. Thus, this research will answer the following questions:
(1) Are the contents designed for training the four language skills presented appropriately in the current junior high school textbooks?
(2) Does the material design meet students' general needs in terms of how to learn and what to learn?
(3) Does the material design meet Taiwanese students' specific needs from the users' point of view?
1.2. Significance of the study
The result of this study hopefully could offer some suggestions to benefit the English language teachers and educators in Taiwan when choosing textbooks. Also, the study should provide modification directions for textbook designers for English textbooks in the future; and last, the study would show the distinct characteristics of different sets of current English textbooks for teachers' reference. Opinions of the teachers and students using different series of textbooks would also be useful for both teachers and the textbook designers in Taiwan.
1.3. Limitations of the study
There are two limitations of this study. First, there was not enough time and money for the researchers to examine all the teaching materials besides the textbooks, such as the teachers' manuals, students' exercise books, or multimedia supplements. The scope of the study is thus limited to the textbooks only, and does not include the accompanying materials. Therefore, future study can take into account the supplementary materials as well, especially workbooks on the different skills like listening comprehension and writing. The second limitation concerns the participants of the study. The raters, teachers, and students who participated in this study all live in Taipei City or County. The samples of the study are therefore restricted to the northern part of Taiwan. In addition, there might be differences in students' experience of learning English and students' proficiency levels between urban and rural areas. Future studies can consult teachers and students in different areas of Taiwan to ensure the information gathered represents widespread opinions.
2. Literature Review
2.1. Language skills
Listening has been considered important in language learning since the late nineteenth century (Rost 2001). During the Reform Movement, linguists regarded the spoken language as the 'source for and means of foreign language learning' (7); therefore, listening became a focus of language teaching. In the 1940s, Bloomfield (qtd. in Rost 2001) declared that by listening to and imitating native speakers, one can comprehend and speak a language. The emergence of the Audiolingual Method came in the late 1950s (Richards and Rogers 2002). Language skills are taught in the sequence of listening, speaking, reading, then writing. This method puts emphasis on oral-aural drills in order to train learners to use language in real situations. In the late 1960s, the Audiolingual Method started to decline, and Communicative Language Teaching (CLT) first appeared. The CLT approach is learner-centered and emphasizes communication and real-life situations. CLT regards listening as a crucial part of communicative competence (Richards and Rogers 2002; Rost 2001). Since the 1980s, listening has been viewed as 'a critical means of acquiring a second language' (Rost 2001: 8).
Reading is the most important foreign language skill, especially in an environment where students have to read English texts for different purposes, but would not necessarily have a chance to speak English (Harmer 1998; McDonough and Shaw 1993). This is the case with the learning environment in Taiwan. The educational policy in Taiwan requires junior and senior high school students to take examinations in order to attend senior high school or college. However, the examinations focus on reading and writing skills in spite of the fact that the curriculum guidelines of language learning request the four language skills to be taught in a balanced way (MOE, Taiwan 2006). Students do not have many chances to listen or speak English outside the classroom. However, they constantly need to practice reading and writing because they have to prepare for tests.
'People may often form judgments about our language competence from our speaking rather than from any of the other skills' (McDonough and Shaw 1993: 151). Needless to say, speaking is the skill that is most apparent when people learn a second or foreign language. Speaking differs from the written language in grammar, word choice, and discourse structures (Bygate 2001). Similar to listening, speaking needs less reaction time than reading and writing, and requires participants to take turns. This is why learners need to take part in speaking activities to practice discussion or problem-solving (Harmer 1998). McDonough and Shaw propose two distinct frameworks of communication in the classroom. One is called the 'pre-communicative speaking skill classroom' (161), in which the form and structure of conversations are controlled. It emphasizes the accuracy of language and is suitable for beginning learners. The second one is called the 'communicative classroom' (161), in which the use of the target language to communicate is emphasized. In pre-communicative language teaching, dialogues are often offered to teach the rules or vocabulary, yet mostly their purpose is to train students to practice grammar structures but not communication. The language teaching environment in Taiwan conforms for the most part to the pre-communicative framework.
Although speaking and writing are both products of language competence and have equal importance, they are quite different from each other. Riddell (2003) defines the differences between writing and speaking. Writing emphasizes accuracy more than speaking does. Furthermore, writing is more economical, in that hesitators such as 'mmm' or 'well,' etc. do not appear in a written language. Speaking and writing are similar in that both skills can be used to express personal feelings, and they can both serve as indicators of a student's language competence.
2.2. Students' needs
The students, rather than the textbooks, are the center of language learning, and textbooks should be helpful for their learning; therefore, when designing teaching materials, learners' needs should be taken into account (Kitao and Kitao 1997; Shao 2002; Tomlinson 2001; Tsai 2001). The MOE's English curriculum guidelines have three goals: to foster students' basic communication abilities, to cultivate students' interests and methods in learning English, and to enhance students' recognition of domestic and foreign culture.
The four language skills are the foundation of communication. Listening and reading are perceptive skills, whereas speaking and writing are productive skills. These skills are different in several aspects, and they are also similar and related in a close way. In real-world situations, language skills are not used in isolation, but in conjunction with each other. Therefore, for textbook editors, the integration of the use of these four language skills should not be neglected when designing textbooks.
Students' motivation is closely related to their academic performance. The best way to increase their motivation is to arouse their interest and to foster their learning strategy. According to Lewis and Hill (1993), decision-makers choosing textbooks commonly ask, 'Is the subject matter interesting?' (50). In Zhu's (2001) opinion, language learning is a long process, and teachers use textbooks to raise their students' learning interests and positive attitude toward English. In addition, the use of learning methods or strategies which can enhance students' knowledge of language use is stressed in TESOL's (1997) ESL Standards for Pre-K-12 Students. Learning methods or strategies are ways for making students learn easier. Both students' interests and their methods (or strategies) toward learning help enhance their motivation to learn. Therefore, in this study, the two concepts, i.e. students' interests and students' method of learning, are seen as one aspect which can enhance their motivation. Textbooks should be able to arouse students' interests toward learning and should include contents related to learning methods or strategies.
Aside from the basic language content, students must be informed about the culture and customs of different countries in order to use their target language appropriately. Native culture and customs should also be included in lessons to enable learners to use English to introduce their native culture to foreigners if necessary.
Furthermore, the specific needs of junior high school students in Taiwan include taking the Basic Competence Test (BC Test) in order to enter senior high school as well as the General English Proficiency Test (GEPT) later on; these needs should also be fulfilled by the English textbooks. Firstly, in the year 2002, the MOE instituted the Senior High School Diverse Entrance Project (referred to as the 'Diverse Admission of Senior High School'), and the BC Test for Junior High School Students is a part of it. Junior high school students can choose their way to enter a senior high school or a vocational school through the result of taking the BC Test. Most of the English teaching classes in junior high school, especially in the third year, focus on preparing students for the BC Test. Therefore, the BC Test is of great importance for junior high school students. Secondly, Taiwan is in the grip of GEPT fever. The GEPT is a test developed for junior high school students or older students and citizens, and it is not suitable for elementary school or kindergarten students (Kidland Information Magazine 2005). Nevertheless, the exam-takers are getting younger. At the elementary level, for example, the level of proficiency is set in accordance with the proficiency level of students graduated from junior high school (GEPT Elementary Level). With English learning, parents have a belief of "the earlier, the better," regardless of the feelings of their children. Therefore, in Lin's report (2003), in the year 2003, 36.6% of exam-takers are between 13-15 years old and 6.9% of exam-takers are still studying in elementary school.
2.3. Textbook evaluation
The English textbook market is now open in Taiwan. There are at least five different publishers that provide well-organized and MOE-approved textbooks. What a teacher can do is to select 'the best possible fit' for his or her students from a variety of choices (Cunningsworth 1995: 5; Yin 1999: 15). Frequently, teachers are expected to be responsible for choosing a good textbook for the students. According to Sheldon (1988), for school administrators the selection of a particular set of textbooks is 'an executive educational decision in which there is considerable professional, financial and even political investment' (237). The cost of unsuitable textbook choices is waste of time, money and even professional knowledge. Furthermore, students nowadays expect textbooks to make their learning easier and enjoyable or they will lose interest quickly (Cunningsworth 1995). Last but not least, the selection of textbooks is closely related to students' academic achievement (Shih 1999), and the wrong choice can impede students' performance. Textbook choices thus have very important implications for schools, teachers and students.
2.4. Textbook evaluation criteria
There are several authors who have provided criteria for English textbook selection, such as Lewis and Hill (1993) who suggest that the items that are interesting for teachers do not necessarily interest students; teachers should put themselves in the students' shoes when choosing materials. Kitao and Kitao (1997) emphasize the inclusion of Standard English and cultural information in the textbooks. Observing the issue from the students' viewpoint, they argue that textbooks should proceed in proper difficulty, have clear instructions and support language learning. Harmer (1998) proposes to the teachers a four-stage procedure, i.e. analysis, piloting, consultation and gathering opinions for selecting textbooks. To analyze means teachers should read carefully through all the different textbook choices and answer questions in a checklist. Piloting means trying to use the textbook in a class. There might be some teachers who teach two or more classes at the same level. They could teach different books and compare them. The consultation stage is important since teachers could seek the opinions of some other teachers who had already used the textbook and see how they evaluate the books. Those teachers' opinions are undoubtedly very valuable. The last stage is to gather opinions from different people, from publishers to bookstore owners, from colleagues to students. Those opinions would be equally valuable in making decisions. Hedge (2000) divides the selecting process into two stages. The first one is to investigate whether the textbook fulfills its stated goal. The second stage is to find out if the materials are appropriate and effective in helping students to learn English. She suggests that for the second stage, it would be more convenient and helpful for teachers to make a checklist according to learner factors, institutional setting, and sociocultural context of their own. After all, those factors vary with different circumstances. Garinger (2001) in his paper discusses the reasons why teachers need to use textbooks and emphasizes the importance of making and offering a checklist for evaluating textbooks.
In this study, all the criteria proposed by the above-mentioned authors is consolidated and adjusted into a checklist for the raters of the study as well as the junior high school English teachers in Taiwan.
2.5. Criteria of textbook selection in Taiwan
Taiwan's educational reform has been carried out since 1997, and the Nine-Year Integrated Curriculum has been implemented since 2001. Educators have different views on the new curriculum because the opening of the textbook market has a direct and huge impact on not only teachers but also students. Some suggestions are also brought up for improvement by different educators.
Some articles and studies in Taiwan offer serious discussion on the issue of textbook evaluation. Yin (1999) lists several basic principles for selecting English textbooks. For instance, teachers need to pay attention to students' needs and the layout and the colors used in the textbooks; whether there are teaching aids or supplementary materials; and other users' opinions should also be considered. Shih (1999) describes the quality of an ideal textbook, in terms of contents, situation, application, supplementation and physical characteristics. Lee (1999) gives some tips for teachers to choose textbooks, such as to see if the content of the textbook follows a gradual progression; if the culture information could be found in daily life; if the expressions in the textbook can be applied in the real language environment; if the pictures attract students' attention; and if the contents are diverse in situation and activities.
Dai (2002) gives some suggestions for the improvement of the MOE's Nine-Year Integrated Curriculum. In addition, he proposes a set of textbook evaluation and selection indicators for elementary school teachers. There are twenty-five criteria and each one would be graded from one to four points. One point means the textbook 'needs improvement' on certain criteria and two points mean it is 'O.K.' Three points will be given to a textbook which is 'good' in terms of the quality of certain criteria. If the textbook is excellent in design, it will get four points. The total score of different series of textbooks can serve as a reference for teachers to select textbooks. The set of textbooks which receives the highest score might be considered to be chosen for the students. Since his checklist is designed for selecting the English textbooks for elementary school, it will not be suitable for the selection of junior high school English textbooks. Therefore, he suggests that teachers of either elementary or junior high schools could modify and utilize his checklist when evaluating and choosing textbooks.
2.6. Related research in Taiwan
Some related research has been done on English textbook evaluation in Taiwan. Ma (2003) conducts an evaluation of the elementary English textbooks of the Nine-Year Integrated Curriculum. She uses American Council in the Teaching of Foreign Languages (ACTFL) Performance Guidelines for K-12 Learners, ESL Standards for Pre-K-12 Students, Association of Language Testers in Europe (ALTE) Framework and the MOE's EAI as the framework to generate a set of textbook selecting criteria--the English Textbook Evaluation Guidelines (ETEG). Her findings show that the organization of elementary English textbooks covers a variety of topics and the design follows the theory of multiple intelligences. The results also indicate that the elementary school English textbooks designed for the Nine-Year Integrated Curriculum show that there is an unequal distribution of the 5Cs--Communication, Cultures, Connections, Comparisons and Communities, and that the textbooks emphasize communication design. Most of the textbook series contains communication and connections, but comparisons and communities are found in only a small proportion. The culture aspect lies somewhere between the two extremes. Though the textbooks are similar in design, each textbook series has its own characteristics. The progression across four levels shows no statistical significance. In addition, several other scholars have reviewed the ways that textbooks are used in elementary schools (Huang 2004; Sun 2000). Wang (2007) uses eight criteria to evaluate three series of textbooks for young learners, Power up English, Darbie, Teach me and English (43). These eight criteria are: appearance, durability and organization, language content, text-types and genres, cultural content, tasks and activities, quality and relevance of illustrations, interest level, and quality and quantity of supplementary resources. She concludes that the design of these books has not fulfilled the national English curriculum guidelines.
Alternatively, in Pan's (2004) thesis, she reviews the tentative versions of the junior high school English textbooks of the Nine-Year Integrated Curriculum. She inspects the distribution of the four language skills by counting the number of activities of each skill. First, she uses Dubin and Olshtain's (1986) communicativeness scale as an instrument to investigate the five versions of junior high school English textbooks. Because the opening up of publication started in 2002, she could collect only incomplete series of textbooks still in the process of revision; therefore, the textbooks examined in the study are called tentative versions of junior high school English textbooks. The researcher tries to find out the communicative degree of the activities in the books. Second, she distinguishes the language skills and their integration into fifteen categories. Activities of the five series of textbooks are classified in terms of the communicative degree and in terms of the fifteen categories. Three raters are involved in the classification. Two randomly chosen lessons from each book are carefully reviewed by the three raters. After classification, the three raters negotiate the differences among them until the final agreements are achieved. The researcher herself then does the rest of the classification according to the agreements among the three raters. She finally computes the degree of communicativeness of each book. She finds that there's an unequal distribution of the four skills and the integrated skills. The listening and speaking skills outweigh the reading and writing skills.
3.1. Materials and participants
Five different versions of the MOE-approved junior high school English textbooks available in Taiwan are being published. Each publisher revises its content every year in order to keep up with modifications of the MOE curriculum guidelines and feedback from teachers.
The materials selection criteria for this study are as follows: first, the textbooks should be compiled according to the MOE's Nine-Year Integrated Curriculum Guidelines; second, the textbooks should be examined and approved by the Textbooks Review Committee of the MOE and should be available on the publishing market now; third, the textbooks adopted in this paper are the most recent complete versions of each publisher. As a result, the textbooks used in this study are published by five well-known publishers in Taiwan. Because our goal is to assess the effectiveness of different types of content rather than to advocate one or another textbook series, the publishers' names are not presented directly in the paper. Instead, they are identified with alphabetical letters, from A to E.
A major caveat here is that only textbooks are examined in this research. Due to limited time and energy, other materials such as exercise books for students, interactive CDs and teachers' manuals were not considered in this study. Each publisher issues two volumes of textbooks each school year (one for the Fall semester and another for the Spring semester). The junior high school level contains three school years, so each series of textbooks has six volumes. Therefore, a total of thirty textbooks were evaluated for this study.
In order to avoid subjectivity, another rater was included in the data analysis process. Aside from the first researcher, who was a graduate student in the Institute of Linguistics at the time of the evaluation, the other rater was a junior high school English teacher with a bachelor's degree in English, who received formal training in teaching education and who had been teaching in junior high school for several years.
The most effective way of knowing about the textbooks is to consult their users' opinions (Cunningsworth 1995), because the teachers and the students are the actual users of the textbooks. Thus, their views toward textbooks are also gathered in this study. Six junior high school teachers and twenty-eight junior high school students who were currently using the five series of textbooks took part in the study's interviews and surveys. They were from junior high schools in the Taipei area. All of the teachers taught English in junior high school for at least three years. Two of the teachers had twenty years of teaching experience. Most of them taught both the unified version of textbooks and also the textbooks designed for the Nine-Year Integrated Curriculum in junior high school. The student-interviewees were randomly selected from these teachers' classes. All of them used the textbooks for at least two years; therefore, they were familiar with the design of the textbooks and could provide valuable opinions. These teachers and students are referred in the paper as T1 to T6 and S1 to S28, respectively.
The first instrument used in this study is the Evaluation Questions for the Junior High School English Textbooks (see Appendix A) which was generated by the researchers after they carefully reviewed and consulted literature concerning language course design. The evaluation questions consisted of seven grading items. Items one to four were related to the four language skills design; the four language skills are the perceptive skills (listening and reading) and the productive skills (speaking and writing). The rest of the three items were concerned with students' needs which are integrated skills, interests and method as well as culture and customs. Under each item, there were several questions for further examination. The evaluation questions were used to evaluate the five sets of present junior high school English textbooks available on the market to see the similarities and differences among each publisher. A rating form was created for the two raters to fill in the scores (see Appendix B). Every volume uses one form. After grading, raters were asked to respond to several open-ended questions and give some comments on each set of the textbooks in another comment form (see Appendix C). Nevertheless, evaluation performed by the raters was not enough. The most effective way to know about the textbooks is to consult the users' opinions (Cunningsworth 1995), because the teachers and the students are the actual users of the textbooks. Therefore, their views toward textbooks are the most direct and valuable information in textbook evaluation. As a result, the third instrument is the Interview Questions for Junior High School Teachers (see Appendix D for the English version; the original instrument was in Chinese). The subjects of the interview were the teachers who were using textbooks published by different publishers of English textbooks. The last instrument of this study is the Survey Questions for Junior High School Students (see Appendix E for the English version; the original instrument was in Chinese). The design of the Chinese versions of the interview and survey was for the convenience of the teachers and students. The design of the students' survey was different from the design of the teachers' interview because students were not as professional as their teachers; therefore, the questions for the students mainly asked about their interests and preferences in regard to the characteristics of textbooks. The interview and survey served the purpose of mirroring students' needs and, alternatively, of strengthening the statistical results of the research.
Four steps were executed in the evaluation of the junior high school English textbooks. The first step was to select English textbooks currently used in Taiwan as the study subjects of this research. Next, the five sets of English textbooks were carefully reviewed. Each set of textbooks from the five publishers was graded, and the raters gave open comments after grading them. The data were processed using SPSS version 10.0 (Statistics Package for the Social Science). Last, the first author of this study conducted interviews with junior high school teachers and administered surveys to the students with the permission of the teachers. The interviews and surveys were done in Mandarin Chinese. The interviews were recorded with permission from the interviewees, transcribed and then analyzed. The students' surveys were also reviewed and dissected. The interview transcripts and survey responses were then translated into English for the write-up of this study. Accordingly, the interviews and surveys constitute the qualitative analysis of the study. "In educational research there is also justification for the use of at least three different viewpoints in analysis" (Burns 2000: 420). Therefore, the method used to enhance the credibility of this study is data triangulation (Berg 1989: 5; Denzin and Lincoln 2000: 391). A cross-reference of the quantitative and qualitative analyses was conducted. Finally, conclusions and suggestions are offered based on the findings of the analyses.
The sources of analysis of this study are the results of the textbook ratings, the open comments of the raters, the teachers' interviews, and the students' surveys.
Inter rater-reliability of the two raters was tested by Spearman's correlation coefficient. The result reveals a coefficient of significant correlation (r = .849, p