Teachers ’ Opinions on the Evaluation of ELT Teachers ’ Books

The aim of this study is to identify differences in opinions on the evaluation of ELT teachers’ books between pre-service and in-service teachers. In literature, it has been argued that the development of teachers’ books for EFL teachers can be assisted by the results of a needs analysis. A total of 65 pre-service and 50 in-servicesecondary school teachers from South Korea took part in a survey. There was a high level of agreement for the 22 criteria-items that asked for the evaluative opinions between two groupsas shown through the correlation.85. However, there were statistically different opinions. The pre-service teachers stressed the importance of clear objectives for each lesson, but the in-service teachers emphasized the need for the inclusion of process tests in teachers’ books. Interestingly, there were also different opinions on the use of a teacher’s book. The results from semi-structured interviews revealed that the former plan to use the teacher’s book for their classroom teaching in the future whereas the latter responded that their own notes would be somewhat more useful than teachers’ books. To conclude, it can be proposed that greater emphasis should be placed on gauging preferences of certain groups of teachers in specific EFL contexts for the development of teachers’ books.


Introduction
In order to evaluate or develop ELT materials for a specific context, it is necessary to identify target users' opinions (Cunningsworth, 1995;Rubdy, 2003;Tomlinson, 2003).Among the target users, along with learners, teachers are the main users of the materials (Breen & Candlin, 1987).Teachers' opinions may influence their own selection of and their classroom use of materials (Crawford, 2002).Therefore, it is, in essence, crucial for materials evaluators or developers to clarifyteachers' preferences on the aspects of materials.Following the theoretical proposals, this study intends to clarify teachers' opinions on criteria for ELT materials evaluation.To be specific, this study focuses on teachers' books and investigates the differences of opinions on them between pre-service and in-servicesecondary school teachers in South Korea.
The value of this study can be supported by the relevance of informationfor teachers in similar ELT contexts and the usefulness of findings for the developers of teachers' books.Wright (1990) states that teachers'opinions in a certain context should be worthy information for teachers in a similar context.Roberts (1996) mentions that "it would be interesting to know whether an "evaluative consensus" exists" in a certain ELT context-and "it would be informative to compare notes" (p.386, emphasis by himself) with other contexts.Tomlinson (2003) indicates that materials developers should utilize the findings about teachers' preferences on materials.Tomlinson and Masuhara (2010) claimsthat the literature on materials development in recent years has presented scholarship and theory but not much empirical investigation.In these aspects, the need for the empirical investigation on teachers' opinions on teachers' books is justified.
The research questions set up for this study are as follows: 1) What are teachers' opinions on materials evaluation criteria for secondary school ELT teachers' books?And 2) Are there any differences in teachers' opinions on materials evaluation criteriabetween pre-service and in-service secondary school teachers?about a particular subject'.Therefore, the term 'teachers'opinions' can be equated with 'teachers'beliefs or ideas'.Clark and Peterson (1986) assert that teachers' beliefs formulate their thought processes.In similar vein, Richards and Lockhart (1994) defines teachers' belief systems as "the information, attitudes, values, theories, and assumptions … that teachers build up over time " (p. 385).In relating to ELT materials, the term 'opinions' covers the notion of 'assumptions'.Woods (1996) mentions that "the assumptions about learning affect the way we assume we should teach the material; and the assumptions about language determine what the material is " (p. 189).In this study, the term 'teachers' opinions' is used as an umbrella term indicating a group of teachers' beliefs, attitudes, values, theories and assumptions.
A number of ELT scholars agree that teachers' responses about teaching materials can be dependent upon their own opinions.Mariani (1980) mentions that many aspects may be taken into consideration in evaluating ELT materials depending on teachers' professional beliefs.Hutchinson (1987) argues that in selecting ELT materials teachers decide the importance of criteria based on their opinions.Lazar (1993) says that teachers may set up own criteria for evaluating ELT materials.Cunningsworth (1995) suggests the idea of applying teachers' own priorities in using a published ELT materials evaluation checklist.Ur (1996) requests teachers think about the important criteria for ELT materials assessment.Tomlinson (2003) points out the need for checking target teachers' opinions on their preferred typesfor the development of ELT materials.Accordingly, it is a valid investigation to ask ELT teachers to give their own opinions on teachers' books.

Studies on Teachers' Opinions on ELT Materials
Empirical investigation about teachers' opinions can be divided into two types: descriptive and exploratory.Descriptive study aims to report a consensus of teachers' opinions, so it is usually conducted in a certain ELT context.For example, Williams (1981) presents a checklist which was made on the basis of a consensus of 36 teachers' opinions for ESP materials from a teacher training course in Manila, the Philippines.Dougill (1987) illustrates teachers' reasons for liking and disliking ELT textbooks in six Oxford Arels-Felco schools.His study reveals that the most frequently mentioned reason for liking ELT textbooksistext-interest.Alderson and Scott's investigation (1992) deals with 50 Brazilian ESP teachers' preferred criteria for ESP materials.The most preferred criterion is interest of student.Tomlinson (1997) reports 24 wishes for ELT materials development collected from 39 teachers, researchers and writers at the conference of Materials Development Association in Dublin.The most frequently mentioned criterion-wish was localization of materials followed by the application of current research into spoken discourse.Stillwell et al. (2010) display theresults of an online survey regarding 13 teachers'opinions about an ELT textbook used in a university in Japan.The results show the need forimprovement of the listening part.All these studiesconfirm that a group of teachers' opinions can be obtained via a survey or an interview and their priorities can be described.The results of this type of descriptive study can be useful for the evaluation and the development of ELT textbooks for a particular context.
The other type is an exploratory study and it generally handles individual differences of teacher variables, such as mother tongue or teaching experience.Not many exploratory studies on ELT materials are found in the relevant literature.One study is Nunan's (1988) report about differences between experienced and inexperienced teachers responses about theclassroom materialsused.He asked 27 senior ELT teachers to rank seven criteria from most to least important.The criteria related to the characteristics of learner-centeredness ranked high in importance.This type of exploratory study attempts to identify causes of differences in teachers' opinions byassuming that different opinions may lead to different decision-making (Woods, 1996), such as selecting or adapting materials.Therefore, the results of the studies will be valuable information formaterials developers who aim to meet the needs of a particular group of teachers.In the present study, both descriptive and exploratory types of studies will be covered.The effect of teaching experience as an individual variable will be the main concern.Other variables, such as the experience of teacher training and first language, are beyond the scope of the present study.
Empirical investigation about teachers' opinions can contribute to the development of ELT materials.In general, studies on materials development can be divided into four types.The first type deals with proposals for developing innovative or effective materials for language teaching.This type of study presents numerous suggestions for designing materials, such as designing appropriate materials for specific skills (Byrd, 1995) or using authentic materials (McGrath, 2002).The second type describes a particular materials development project, such as Brunei and British teachers' cooperating on a textbook development project (Leburn, 1991), and the Rumanian textbook project (Popovici & Bolitho, 2003).Richards (2001) depicts textbook writing procedures for Spring Board.The third type illustrates ways of administrating materials development.For example, Block (1991) proposes a way of developing materials within a single institution viateachers' cooperative work.Prowse (1998) describes how materials writers develop ELT materials.The fourth type suggests the adaptation of findings from empirical studies (Tomlinson, 2012;Tomlinson & Masuhara, 2010).It advocates the reliability of evidence from empirical studies as a basis for materials development.Among these different types, this study focuses on the fourth type as an appropriate way for developing ELT teachers' books.

Studies on ELT Teachers' Books
In the field of ELT materials evaluation or development, not much attention has been paid to teachers' books.Coleman (1985) and Cunningsworth (1995) directly handle the issue of ELT teachers' books.Coleman (1985) considers teachers' books as a very important assistant for classroom teachers, especially for nonnative English teachers and novice teachers, and stresses the needs for relevant studies.He presents five criteria for the evaluation of teachers'books, which include the appropriateness of proposed language teaching methodology, the specific explanations for the contents, the considerations of cultural aspects, the guidance for assessment, and the efficiency of presentation.Cunningsworth (1995) claims that teachers' books are very influential materials for English language teaching.He presents 22 criteria for the evaluation of teachers' books-which will be adapted as question-items of the survey instrument in the present study.
Atsecondary schoolsin South Korea, as a target context in this study, textbooks are subject to evaluation by the ministry of education.The government gives certification for nation-wide use of certain textbooks after knock down evaluation.However, teachers' books are not subject to evaluation.In case of primary school teachers' books, they are evaluated by the government for certification.The three main evaluation criteria regard organization, the explanation of teaching methodology and assessment, and the accuracy and balance of the contents (KICE, 2011).As secondary school teachers' books are considered as supplementary materials by publishers and materials developers, there has been a lack of relevant studies.Kim (2013) analyzesa total of 155 published articles in nine leading local journals published from 2000 to 2011 in South Korea.The results show that the most frequently covered area of the study is analysis of materials and follows design and selection of materials.Only five studies deal with the issue on EFL teachers.These studies do not specifically mention how to assess information on teachers' opinions in relating to teachers' books.In other words, the literature on ELT materials evaluation or development does not offer us much to consider about teachers' books.It justifies the need for the present empirical study.
The review on the relevant literature has led us to make the following justification for the present study.First, the preferences to materials evaluation criteria can differ according to different contexts, soit should be better to investigate teachers' opinions on criteria for teachers' books within a single context.Second, there can be individual differences in teachers' opinions in terms of the length of teaching experience, so it should be valuable to identify the differences between pre-service and in-service teachers with regard to teachers' books.Third, different data collection methods can be used in order to investigate the teachers'opinions in depth.A questionnaire in the form of a checklist and semi-structured interviews can be employed.

Setting
The target context for the present study isan English class at secondary schools in South Korea.English is a compulsory subject from primary school to sophomore year at most universities in South Korea.At secondary schools, students usually take three to five hours of English classes per week.In general, both English and Korean are the mediums of instruction in the course.There are usually 30 to 40 students in a classroom.The development of communicative ability is mainly emphasized.As English is rarely used for social interaction in South Korea, most students have had little opportunity to use English outside the classroom.However, many students attend private institutes for extra classes in English.
The secondary school English textbooks are certified by the ministry of education.They are developed by Korean secondary school teachers and university professors of English and published by local publishers.The in-house secondary school English textbooks are designed to follow the requirements of the national curriculum, in the aspects of level, balance of skills, methodology and so on.The certified textbooks generally have 12 lessons andinclude cross-cultural issues between Korean and Western cultures.They reflect needs and interest of Korean students, and deal with linguistic problems in considering students'past learning experiences in primary school.Therubrics and explanations are written in Korean.Teachers' books are usually made after the main textbooks obtain the government certification.

Participants
A total of 115 English teachers participated in the questionnaire survey.All of them were Korean and non-native teachers of English, and 31 (27%) were male and 84 (73%) were female.Among them, 65 were pre-service teachers and 50 were in-service teachers.The number of the in-service teachers, as middle school, high school and university teachers, was respectively 17, 26 and 7.Those who were teaching at university had had teaching experience at secondary schools.The average length of English teaching was 10.4 years, ranging from 1 and half years to 26 years.In-service teachers were teaching at institutions located in 12 different cities in South Korea.Twenty-two (19%) were teaching at institutions in the capital city, Seoul.Sixty-five pre-service teachers who wanted to be secondary school teachers in future were being trained at a university located in Seoul.Of 65 pre-service teachers, 19 were involved in qualitative data collection.They were the teacher-trainees enrolled in a university course titled ELT materials and methods.To check their opinions about teachers' books, semi-structuredinterviews were carried out.Three professional textbook writers were also invited for semi-structured interviews.All of them were university professors with secondary school teaching experience.One writer published primary, middle and high school English textbooks in South Korea.Other writers worked on primary and middle school English textbooks.

Instruments
In this study, a questionnaire and a semi-structured interview were used as data collection instruments.The questionnaire consisted of three parts eliciting personal backgrounds, opinions on the 22 criteria-items for teachers' books evaluation, and their opinions about the use of teachers' books.The 22 criteria-items in the questionnaire were directly adapted from Cunningsworth's (1995) criteria.The questionnaire itemswere translated and presented in Korean to reduce the response time of Korean teachers of English.Under each item-which will be presented in next Results and Discussion part, the respondents were asked to rate their opinions.Ratings were on a 5-point Likert scale on which 1 = not at all important, 2 = not so important, 3 = important, 4 = very important, and 5 = extremely important.In the introductory part, the aim of the survey was said to be to elicit opinions on ELT teachers' books evaluation criteria.The respondents filled out the questionnaire anonymously.Onepage of A4 wasused vertically to allow them to see all questions once.The request, writing down qualitative comments, was placed at the very end of the questionnaire.It took eight months, from October 2013 to May 2014, to get back all the questionnaires.The questionnaire for pre-service teachers was administered in five different teacher-training classrooms.This investigator as a teacher-trainer distributed and collected the questionnaires.The questionnaires for in-service teachers were collected during several meetings organized by the Korea Institute for Curriculum and Evaluation.The internal consistency reliability, i.e. the item reliability, of the 22 items representing evaluation criteria for ELT teachers' books was estimated.Cronbach's alpha was chosen as a measure.Item reliability was α = .83,indicating a high level of internal consistency in the respondents' ratings on the items.
The semi-structuredinterviews with 19 pre-service teachers were carried out by this investigator.At first, they were asked to write one lesson of a secondary English textbook as a part of a courseproject.Groups consisted of three or four student selected members.They could refer to the published secondary textbooks as a reference for the materials design.The amount of time for writing the materials was one month.They submitted one textbook, one workbook and one teachers' book.The focal interview-question asking their opinions was: "What do you usuallyconsider in writing a secondary school English teachers' book?"The interview-question for the three professional textbook writers was identical to the question asked to the pre-service teachers.The investigator prepared an interview notebook to record their answers.

Data Analysis
The collected quantitative data were analyzed by SPSS 17.0.Descriptive statistics of mean scores and standard deviations of the 22 individual criteria-items for teachers' books evaluation were calculated.The mean score was regarded as the level of importance on that criterion.The group's preferences were identified by the ranking order of mean scores.For inferential statistics, a correlation test (Pearson r) was conducted to check thelevel of agreement of opinions betweenthe pre-service and in-service teacher groups.To check statistical differences in opinions, 22 independent t-tests for the individual items were carried out.The teacher-group was considered as an independent variable, and the rating as a dependent variable.A factor analysis was employed to determine how many factors could be defined under 22 criteria-items.The principal component analysis via a varimax rotationwas selected to extract underlying factors.In case of the opinions about the use of teachers' books, a Chi-squared test was adopted to check the statistical difference in frequency of responsesbetween the pre-service and in-service teacher groups.

Opinions on Teachers' Books Evaluation Criteria
The first research question in this study is 'what are teachers' opinions on materials evaluation criteria for secondary school ELT teachers' books?'To answer this question, the mean results of the ratings on the 22 evaluation criteria werechecked.The second research question is 'are there any differences in teachers' opinions on materials evaluation criteria between pre-service and in-service secondary school teachers?'The measure used to answer this question was an independent t-test.The results are presented in the following Table 2. * M = mean, SD = standard deviation, R = rank.
Among the 22 criteria, the criterion of 'outline plans for each lesson' (item 11) was ranked the most important.The question was presented as 'Are there outline plans for each unit/lesson?'The second most important was 'contents relevance to the textbook' (item 16).It was presented as 'Are the contents of the teacher's book related easily to the relevant sections of the student' book?' The criterion of 'comprehensiveness and flexibility' (item 1), read 'Is it comprehensive and flexible?'It was ranked third.The fourth most important was 'learning difficulties prediction' (item 13), and read 'Are learning difficulties predicted and appropriate advice given?' Next followed the criterion of 'detailed information and advice' (items 14), read 'Is the information detailed and advice given?' The results show that the teachers in the survey regarded the criteria related to comprehensive coverage of information for lesson plans and relevance to the main textbook as relatively more important in comparison with other evaluation criteria.It seems probable that the teachers might have had a reliance on the main textbook and considered the teachers' book as a useful supplementary resource for classroom lesson.
The criteria considered moderately importantwere 'contribution to learner motivation' (item 20: Does it make a positive contribution to heightening and sustaining learner motivation?), 'keys to exercises and activities' (item 18: Are keys to exercises and other activities provided?),'teaching procedures explanation' (item 12: Are teaching procedures clearly explained?),'language items explanation' (item 10: Are new language items explained intelligibly in terms of their form and meaning/use?), and 'comprehensiveness to teachers' (item 2: Is it written so as to be comprehensible to less experienced teachers?).They were ranked as the sixth to the tenth.The results imply that the teachers seemed to seekto gain ideas for leaner motivation via teachers' books.As learner motivation may have an effect on languagedevelopment, it seems natural that the teachers were keen on the motivating factors presented in teachers' books.The teachers also expressed a relatively higher importance on the criteria related to the keys to the main textbook and the coverage of teaching techniques and language items, especially for novice teachers.
The criterion of 'cultural explanation' (item 7: Is there enough cultural explanation?) was regarded as slightly less important than the criteria above.Itwas ranked eleventh.This is likely due to the fact that nowadays, cultural explanation may be obtained not only from teachers' books but also from Internet resources.The criteria of 'same detail given for every lesson' (item 15: Is the same detail given for every unit?), 'clear objectives for each lesson' (item 9: Are there clear objectives for each unit/lesson?) and 'guidance on regular process tests' (item 19: Are there regular process tests and advice on when and how to use them, and how to follow them up?) were also considered as slightly less important.They were ranked as the twelfth, the thirteenth and the fourteenth.The practical reasons for theregularadministration of class achievement testsmight have led the teachers to give a slightly higher priority to the criterion about process tests.On the comment section in the questionnaire, some teacher-respondents specifically mentionedthat it was difficultfor them to makenew test items regularly, so they needed assistance from teachers' books.
The teachers did not rate highly the criteria related to 'feedback functions'.The criteria of 'guidelines for self-evaluation' (item 22: Are there any guidelines for evaluating how well lessons went?), 'advice on monitoring/correction' (item 17: Is advice given on informal monitoring of students and on using correction techniques?), and 'encouragement to own ideas' (item 21: Are teachers encouraged to note their own ideas in it?)were ranked as the fifteenth, the nineteenth and the twenty-second.Perhaps the teachers thought that the teachers' books were intended to be supplementary tothe main textbook, not for their own self-training.This might have resulted in this pattern of the responses.
Other low rank criteria were 'clarity and explicit of approach' (item 4: Is the underlying approach of the writers expressed clearly and explicitly?), 'guidance on the teaching procedures' (item 6: Does it provide enough guidance on the teaching procedures advocated?),'prescriptive guidance' (item 8: Is the guidance prescriptive?),'information on the language items'(item 5: Does it provide enough detailed information on the language items to be taught?)and 'suitability for non-native teachers' (item 3: Is it suitable for non-native teachers?).They were ranked as the sixteenth, the seventeenth, the eighteenth, the twentieth, and the twenty-first.Presumably, the teachersassumed that too much explicit guidance or information in printed teachers' books would be impractical.One teacher commented in the questionnaire that economy of information was highly valued asnot many teachers wanted to carry a heavy teachers' book.The low level of importance of thesecriteria seems also reflect the role of teacher as a busy classroom manager who might need to handle both the main textbook and teachers' book at the same time.
The overall results indicate that both groups generally agreed to the degree of importance on the questions about evaluating ELT teachers' books.To check the level of agreement between the pre-service teachers and the in-service teachers, the correlation for the mean ratings on the 22 criteria was calculated.The correlation was computed as r = .85(n = 22 items), indicating a high level of agreement between the two groups.However, the criteria of 'clear objectives for each lesson' (item 9) and 'guidance on regular process tests' (item 19) revealed statistical differences in responses between pre-service and in-service teachers.Pre-service teachers gave statistically higher scores on the criterion of 'clear objectives for each lesson' than in-service teachers.It can be inferred that the in-service teachers might have had anexplicit perspective on their classroom teaching objectives on the basis of their experiences.They might also have had a fairly high level of confidence in their skills to adapt the main textbook by referring to the course objectives.On the contrary, it seems probable that the pre-service teachers might have had a low level of confidence, so they were more enthusiastic at identifying clear lesson objectives throughout teachers' books.In case of the criterion of 'guidance on regular process tests', in-service teachers responded statistically higher than pre-service teachers.It seemed to be caused by the possibility that in-service teachers needed to make new test items every year although they had taught from the same textbook.They could not re-use the test items, so they needed lots of test items for regular achievement test.
Due to this necessity, in-service teachers might have provided answers at a statistically higher level of importance on the criterion than pre-service teachers.

Factors of Opinions on Teachers' Books Evaluation Criteria
To identifysubsets of the teachers' opinions that were mutually highly related, a factor analysis was carried out.The logic behind using the factor analysis was that the teachers who gave a high or low ratingon one of the criteria also tended systematically to give a high or low ratingon the other related criteria, so these criteria could be grouped as one head-criterion.The use of a 5-point interval scale on each criterion-item met the requirement for factor analysis.The results of the analysis are presented in Table 3.
3) suitability for non-native teachers .700 4) clarity and explicit of approach .8365) information on the language items .812 6) guidance on the teaching procedures .667 * Factor loading below .39 is not presented.
The factor analysis revealed that the 22 items could be grouped into seven factors.These seven factors explained 63% of the results.Items 16 and 18 were identified as one factor and could be called the factor of 'relevance to the students' book'.Items 1 and 2 were revealed as another factor of 'comprehensiveness'.Items 9, 10, 11, 12 and 13 were related to a factor of 'ideas for lesson'.Items 14, 15, and 19 were connected with a factor of 'detailed information'.Items 7 and 8 dealt with a factor of 'cultural explanation'.Items 17, 20, 21 and 22 wererelevant with a factor of 'feedback functions'.Items 3, 4, 5 and 6 were regarded as a factor of 'enough information'.These factors could be considered as constructs in this group of teachers' responses.
To check the the teachers' opinions on the head-criteria for evaluation of secondary school ELT teachers' books, the mean results of the ratings of the 7head-criteria werecalculated.The score of one head criterion was obtained by the mean of the relevant criteria.To examine the differences in the two groups' opinions on the head-criteria, independent t-tests were conducted.The results are shown in Table 4.For both groups of teachers, the head-criteria of 'relevance to the students' book' and 'comprehensiveness' were considered highly important.The head-criterion of 'ideas for lesson' was considered moderately important.All threeof these head-criteria received mean scores over 4.00.Like the results of analysis on individual evaluation-criteria, the teachers' opinions showed ahigh level of importance on the comprehensive coverage of ideas for lessons relating to the students' textbook.With regard to the head-criterion of 'detailed information', which included the criterion of 'guidance on regular process tests', there was a statistically significant difference in ratings between pre-service and in-service teachers, favoringthe in-service teachers.Perhaps the in-serviceteachers thought the detailed information about regular test items to be practical and useful for them.The teachers did not rate highlythe head-criteria of 'cultural explanation', 'feedback functions', and 'enough information'.The teachers might have regarded these head-criteria as minor elements in teachers' books.This pattern of results is similar to that from the analysis on the individual criteria.Therefore, in further relevant or replication studies in other ELT context, the minimized version of the questionnaire, adapting the head-criteria factors as question-items, may be developed and used.

Opinions on the Use of Teachers' Books
The teacher-respondents were asked to give their opinions on the use of teachers' books.For the pre-service teachers, they were asked about their expectations for future use.The in-service teachers answered according to their current use.For a statistical test, a Chi-Squared test was carried out.The results indicate that there was a statistically significant difference in the frequencies of responses on the use of teachers' books between pre-service and in-service teacher groups, χ2 = 25.76,p < .05(=.00).The pre-service teachers seemed to be more dependent upon teachers' books than the in-service teachers.The majority of pre-service teachers (n = 48, 74%) responded that they would always or often use teachers' books, whereas the majority of in-service teachers (n = 33, 66%) answered that they sometimes or seldom used them.Their qualitative comments revealed two specific reasons.First, the experienced teachers were not satisfied with the current teachers' books.When they had been novice teachers, they had often used them.However, as they becameexperienced, they accumulated their own supplementary materials and the need for teachers' books gradually decreased.Second, the pre-service teachers were not confident about their own teaching skills.They usually assumed that they would receive a lot of help from teachers' books upon beginning their teaching practice.The results imply that in designing teachers' books, materials developers may need to specify their target teacher groups in terms of the length of teaching experience.Developing two different versions of a teachers' book for novice teachers and experienced teachers can be one of the possible ways to meet each group's needs.

Qualitative Comments
Qualitative comments were collected from the survey respondents, three professional textbook writers and 19 pre-service teacher-trainees.The writers and teacher-trainees took part in a semi-structured interview.

Comments from Survey Respondents
Many survey respondents wrote some comments.They could be categorized as follows;their own criteria for teachers' books evaluation, their reasons for the use of teachers' books, criticism orapproval of teachers' books, description of current situation about the use of teachers' books in Korean context, and recommendation of particular textbooks and teachers' books.In this part, some of their comments are presented: The teachers' books should provide skill-integrating activities that develop all four language skills.
A content-based approach is good for secondary school English teaching, and the methodology should be outlined in teachers' books.
The techniques for learning strategy training need to be included.
The ways to present grammatical points, vocabulary and other new structures in a meaningful context should be offered in teachers' books.

How about including cultural values that are appropriate to handle in a formal classroom setting?
What is important in teachers' books is providing practical tips to reduce students' classroom anxiety.
There should be supplementary materials appropriate for intellectual challenges, sound morality and positive attitude.
Teachers' books should be handy to carry and easy to copy.
The teachers' books should be well-edited: the text on the left hand page and teaching guide on the right hand page.
We have to admit that the written book cannot provide enough exercises for listening practice.We need audio teachers' books.
If the main textbook is good enough, the need for teachers' books will decrease.
A teachers' book may ruin teachers' imaginative approach to the text.

Comments from Textbook Writers
Professional textbook writers expressed their focal points in writing teachers' books, when they were asked the question "What do you usuallyconsider in writing a secondary school English teachers' book?" Writer A mentioned thathe had tried to provide in-depth information about language expressions, supplementary explanations about culture and grammar, specific ideas for lesson planning, and various samples of classroom English instructions.Writer B focused on the use of a variety of vocabulary, the additional information for teaching procedures, and the inclusion of numerous test-items.Writer C made an effort to offer practical activities and many classroom English instructions.He also said that he had wantedhis teachers' books to be user-friendly.

Comments from Teacher-Trainees
The interview-question for the 19 teacher-traineeswas identical to the question above.They were requested to write one lesson fora secondary English textbook.In fact, they were focusingonthe adaptation of the current secondary textbooks, rather than the creation of a new textbook.They answered that in writing a teachers' book they had trouble in explaining the specific reasons for choosing the topic for the reading part of the main textbook.They also said that it was difficult for them to give reasons for integrating skills and selecting new vocabulary and new grammar.They seemed to believe that teachers' books should deal with the issue of skill integration in-depth, be helpful in reducing students' fear of making mistakes, and enhance learner motivation through proper guidance on the chosen topic.

Conclusion
This study aimed to answer two research questions about teachers' opinions on materials evaluation criteria for secondary school ELT teachers' books and differences in opinions between pre-service and in-service secondary school teachers.A total of 115 pre-service and in-service teachers from South Korea participated in a questionnaire survey.The teachers attributed greater importance to the criteria related to comprehensive coverage of information for lesson plans and relevance to the students' textbook.The teachers did not rate highly the criteria related to feedback functionsand abundance of information.Between the two groups, there was a high level of agreement regarding to the 22 criteria-items for evaluating teachers' books.However, there were statistically significant differences in the criteria of 'clear objectives for each lesson' and 'guidance on regular process tests' between them.Also, there was a significant difference in the use of a teachers' book.The pre-service teachers were likely to be willing to get help from teachers' books whereas the in-service teachers seemed to have a high level of confidence in their classroom language teaching and their ability to adapt a textbook without a teachers' book.For in-service teachers, test items seemed to be useful resources from teachers' books.A factor analysis revealed that there could be seven factors, which may be adapted ashead-criteria factors for further studies.Qualitative comments from the survey respondents, three professional textbook writers and 19 pre-service teacher-trainees were supplemented for the findings of the quantitative analysis.
Based on the results, the followings can be suggested.First, for the development of user-friendly teachers' books, we have to make an effort to reflect preferences of certain groups of teachers in a specific EFL context.In particular, for novice teachers in secondary schools in Korea or in similar contexts, a teachers' book should provide a variety of teaching techniques, suggestions for very detailed lesson planning with specific guidance, and clear objectives for each lesson.For experienced teachers, the efficient way of presenting information and the inclusion of lots of test items should be well-recognized by materials developers.Second, a workshop for designing an ELT teachers' book may contribute to understanding its characteristics, and so will be useful for teacher-trainees to use it in an appropriate manner.As Canniveng and Matinez (2003) point out writing ELT materials can work toward teacher development.To educate teacher-trainees, teacher trainers need to set up an appropriate task of developing a teachers' book.Third, for the improvement of teachers' books for a particular group of teachers, it might be necessary to develop teachers' books based on empirical investigation rather than trends or publishers' needs.Possible research agendas may include the identification of differences in opinions between native and non-native teachers of English (Skierso, 1991), the investigation of changes in teachers' opinions before and after materials development training, the comparison of the test items presented in teachers' books andtest preparation materials (Bailey & Masuhara, 2013), and the evaluation of particular teachers' books based on the responses after actual classroom use (McDonough, Shaw, & Masuhara, 2013).Theresults of the investigations will be useful references for developers of ELT teachers' books, in order to designappropriate ones or, at least, to avoid discrepancies between what materials users want and what materials developers supply.

Table 1 .
Overview of teacher-participants by group, gender, teaching location, and teaching experience *Others (in terms of teaching location) include 12 different areas in Korea.

Table 2 .
The results of teachers' ratings on the importance of evaluation-criteria for teachers' books

Table 3 .
Factor analysis of the teachers' opinions on evaluation-criteria for teachers' books

Table 4 .
The results of teachers' ratings on the importance of head-criteria for evaluating teachers' books

Table 5 .
Frequency of opinions on the use of teachers' books