A Coursebook Evaluation

Ramin Rahimy (Ph.D. Student of TESOL)

Islamic Azad University (Khorasgan Branch)

Contents

Abstract

1. Introduction

2. Evaluation framework

2.1. Curriculum Vs Syllabus

2.2. The university systems in Iran

2.3. General & specific issues on Reading skill

3. Evaluation

3.1. Macro Evaluation

3.2. Micro Evaluation

3.3. The Compatibility with the Iranian curriculum

4. Conclusion

5. Appendix

ABSTRACT

Textbook evaluation is one of the most significant decisions facing a foreign language teacher who is in the position to be given the responsibility of teaching a foreign language to a given group of students. There are many factors to be taken into account when evaluating material for use with speakers of other languages. According to Nunan (1988), materials are, in fact, an essential element within the curriculum, and do more than simply lubricate the wheels of learning. In the present study, a textbook on the Reading Comprehension skill has been evaluated based on a number of criteria mentioned in the report. The material is a teacher-made one designed for both general and specific use by university students. Also, the compatibility of the content of the material with the Iranian curriculum/syllabus has been discussed.

1. INTRODUCTION

This paper deals with the evaluation of an ELT coursebook named Reading

Comprehension, for the University Students. The ability to evaluate an ELT coursebook effectively is a very significant one. First, most teachers use them during their teaching career and in contexts where they have a choice; this choice should be on the basis of clear theoretical and applied (practical) considerations. Second, such an evaluation not only facilitates the selection of new materials, it can also be used to reconsider old ones on a regular basis. This paper seeks to reconsider the use of Reading Comprehension especially by examining its compatibility with the current state curriculum as well as my teaching experience.

Section 2 clarifies the ground by first defining some terms such as curriculum and syllabus, which prominently figure in various coursebook evaluation checklists. It then briefly outlines the context of this evaluation, namely the university system, and its curriculum. In section 3, I will discuss the use of a two stage evaluation model consisting of a macro, and a micro evaluation. As the discussion of the outcome of the evaluation, I will tackle some important issues such as the role of authenticity in ELT teaching materials. The section finally concludes with a proposal for either reusing and adapting the coursebook or choosing to substitute another one instead. In section four, the general conclusion of the study will be mentioned in which discussions, comments, as well as a number if suggestions for further research will be represented.

What is important here is that bringing the concept of curriculum into controversy is not so easily done, particularly when the curriculum of a specific system of education is under debate. Since, in the present study, the Iranian university systems have been analysed, I have made an attempt to stick to the most prominent points probably relevant to the study. Hence, the results of the study may or may not be generalisable to other researches or studies. The main concern of the following section is the Evaluation Framework of the study.

2. EVALUATION FRAMEWORK

2.1) Curriculum vs. Syllabus:

At the heart of the educational enterprise is the educational programme offered. Until fairly recently, most educational authorities have considered the syllabus to be the educational programme(Johnson 1989, p:25-26). It is the syllabus which has received the most attention in educational design and implementation. Consideration of syllabus change as but one element in a constellation of related elements is a fairly recent phenomenon in discussions of educational renewal. This larger view of educational planning has been often labelled as curriculum development. The assumption has been that syllabi and curriculum are synonymous. Syllabi, which prescribe the content to be covered by a given course, form only a small part of the total school programme. Curriculum is a far broader concept. It refers to all those activities in which learners engage under the auspices of the school. This includes not only what learners learn, but how they learn it, and how teachers help them learn, using what supporting materials, styles and methods of assessment, and in what kind of facilities (Rodgers, 1976). According to White(1988, p:4), the main difference between curriculum and syllabus is that the former specifies the totality of content for all school subjects taught in a particular type of school, and the latter indicates the content of a specific school subject. Stern(1983), also, holds that there is a multidimensional curriculum comprising at least four categories of objectives. The first and the major objective is proficiency in the second language. The second one is knowledge which comprises an explicit knowledge about the second language(L2) and knowledge about the corresponding culture. The third objective expresses the belief that the cultivation of affective objectives forms an integral part of the scheme. This objective includes values and attitudes related to the language and culture. A final behavioural category, which is described as transfer, acknowledges as an objective the possibility of learning a particular language with the purpose of generalising beyond the language in question. This objective repeats the three L2-specific objectives on a more general plane and specifies the aim to learn a second language in such a way as to learn about language learning in general, in terms of (a) proficiency, i.e., skill in language learning, (b) conceptual knowledge, as well as (c) more generalised values and attitudes.

The content categories are broadly conceived as language, culture, communication and general language education. Language implies the particular L2, culture, the target culture or several target cultures(for example, the culture of France and/or French-speaking Africa and/or French-speaking Canada; the culture of Britain and/or English-speaking North America or Australia, etc.). Communication refers to activities in the language or suggest content other than the language itself which engages the learners as a participant in communication with speakers of the second language either directly or vicariously. Finally general language education is that content beyond the particular second language and target culture which will enable the learner to go beyond the language given.

2.2) The University Systems in Iran:

In order to enter a university programme in Iran, the candidates (usually high school junior students) after receiving their Diploma should take part in a competitive exam traditionally named the University Entrance Exam here the UEE to prove that they possess enough ability and knowledge. In the Iranian education system, high schools are affiliated to and managed under the supervision of the Ministry of Education of Iran. Primary Schools as well as Intermediate ones [Rahnamaee] are also managed by the Ministry of Education, but they are not going to be elaborated on in the present study. University activities, in Iran, are under the supervision of the Ministry of Science, Research, and Technology. These activities are based upon at least three systems namely: the State Universities, the Islamic Azad University, and the Payam-e-Noor (Distance Learning) University. All such universities provide their candidates with special handbooks in which course and programme specifications have been detailed. Then, after the candidates admittance is declared, they will keep pace with the regulations of each system. The three university systems are explained as follows:

          The State Universities (SU):

Group 1 (SUG 1):

* Registering and selecting course credits

* Compulsory class attendance and lecture sessions(16 weeks)

* Taking quizzes and mid-term exams (the 8^th week)

* Taking final exam

* Paying no tuition

Group 2 (SUG 2):

* Registering and selecting course credits

* Compulsory class attendance and lecture sessions (16 weeks)

* Taking quizzes and mid-term exams (the 8^th week)

* Taking final exam

* Paying tuition

The Islamic Azad Universities (IAU):

* Registering and selecting course credits

* Compulsory class attendance and lecture sessions (16 weeks)

* Taking quizzes and mid-term exams (the 8^th week)

* Taking final exam

* Paying tuition

                  The Payam-e-Noor (Distance Learning) Universities (PNU):

* Registering and selecting course credits

*No compulsory class attendance and lecture sessions (16 weeks)

* Taking quizzes and mid-term exams (the 8^th week)

* Taking final exam

* Paying tuition

As mentioned previously, to enter each of the three university systems, the candidates take part in a nationwide competitive exam. There is one exam to single out the qualified candidates for SUG 1, SUG 2, and PNU. Another separate exam makes the selection of the qualified candidates for IAU.

The point to consider is that the systems under discussion follow an identical curriculum: the curriculum has been planned under the auspices of the High Council of the Curriculum Planning Department in the Iranian Ministry of Science, Research, and Technology. The study reported here is based on the English Language Translator Training Programme (ELTTP) curriculum at B.A. level. Thus, what follows next, focuses on the specifications of the study in terms of the ELTTP.

2.3) General and specific issues on the Reading Comprehension skill:

Before bringing the overriding issue of the reading comprehension skill into discussion, let us have a brief focus on the definition of the skill to be able to evaluate and make judgement about the current objectives the Iranian Ministry of Science, Research, and Technology has issued in this respect.

  The Reading Comprehension Skill (Definition):

Reading is one of the receptive skills of language. It is a process of involving the activation of relevant knowledge and related language skills to accomplish an exchange of information from one person to another(Chastain 1988, p:216). It has occasionally been called a passive skill because the reader does not produce a message like a speaker, but it is not as such: reading is an incredibly active occupation. To do it successfully, learners have to understand what the words mean, see the pictures the words are painting, understand the arguments, and work out if they agree with them, otherwise, they only just scratch the surface of the text, and they quickly forget it. There are two types of taxonomy to categorise the process of reading comprehension skill: the first taxonomy divides the skill into reading aloud consisting of the enunciation of the sounds symbolised by the printed or written marks on the script by the learner in a conventional way, and reading comprehension in which the learner must be able to derive meaning from the word combinations in the text, and to do this in a consecutive fashion at a reasonable speed, without necessarily vocalising what is being read. In fact, in comprehension, the goal is to read for meaning or to recreate the writers meaning. The second taxonomy divides the skill into: skimming(to get the general idea of what the text is about), scanning(to read the text for particular bits of information they are searching for), extensive reading(to read for pleasure toward the top of their reading speed), and intensive reading(to read slowly for information especially complicated material). Care must be taken that the presentation of language through reading passages(with appended comprehension questions) is a well-established and very familiar pedagogic practice(Widdowson 1978, p:77). One major problem concerning the reading comprehension skill is the problem of the authenticity of the materials. Although one normally thinks of authentic texts as those written for native speakers, Byrnes(1985) states that due to problems students have with such texts because they are unfamiliar with the culture, one may think of the material by native speakers for language students as being authentic. Generally, any text that an author writes to be able to communicate some message is authentic because it has an authentic purpose that it conforms to authentic language use. The above-mentioned considerations are needed here to evaluate the material of the study which consists of a number of passage extracts with appended comprehension questions.

  The Iranian curriculum/syllabus (objectives for reading comprehension):

The objectives of a course of reading comprehension skill in an Iranian university ELTTP are as follows:

1.     Improving students comprehension ability

2.     Increasing students reading speed

3.     Enlarging students ability of perceiving word meaning from context

4.     Guessing meaning from context and contextual clues

5.     Understanding implicit as well as explicit meaning

6.     Skimming

7.     Scanning

Also, as a partial fulfilment, the trainees are required to read about 300 pages on a

topic and submit their summarisation protocols to the teacher orally or in a written form. This is a part of the reading comprehension syllabus. There is no compulsory textbook or pamphlet recommended by the syllabus itself, however, there is a significant number of sources to be used as the material of the course. Some of the materials are teacher-made in the sense that they have been devised by the teachers based on the teaching conditions and environment. One of these teacher-made materials is the focus of the present study. The material(book) has been designed for the three courses of: Reading Comprehension(1), Reading Comprehension(2), and Reading Comprehension(3). What follows will be the evaluation of the content of the book under discussion based on the elements elaborated on.

3. EVALUATION:

Three sections cover the practical evaluation report of this study: section one and two will cover a macro and a micro evaluation of the book, then in section three the compatibility of the book with the curriculum/syllabus planned by the Iranian Ministry of Science, Research, and Technology will be debated. The specification of the book can be found in the Appendix section at the end of the article.

3.1) The Macro Evaluation:

3.1.a) Contents:

                 What kind of a syllabus is the book based on?

Forasmuch as the passage extracts have been arranged from easy to difficult, the book seems to be based on a linear syllabus; however, it is not impossible to go through the extracts randomly so a cyclic syllabus can be said to be true for the book. In fact, there is a general overlap between the two.

                   What is the organisation and the connection of the units like?

The book contains 60 extracts, 20 of which have been planned for Reading Comprehension (1), 20 for Reading Comprehension(2), and 20 for Reading Comprehension(3). Each extract is accompanied with itself a number of multiple-choice questions as exercises.

                   What claims are made about the book approach and methodology?

The writers of the book believe that the passages they have extracted help the learners who intend to strengthen their reading comprehension skill for academic, personal, or career purposes. Furthermore, they consider the book a great help for the university students who are attending the Reading Comprehension courses that were mentioned previously.

3.1.b) Layout:

                   Does the table of contents facilitate orientation?

There is no systematic table of contents as such in the book. One reason can be that the passages have been included topic-free for the purpose of guessing the main idea or choosing a suitable topic for the extract.

                   What visual material does the book contain and does it help in the learning process?

The book contains no picture or other visual material, so students are exposed merely to the orthographic content. No clear reason or justification has been explained in the introduction of the book. It can be guessed that the writers intend to force their students to be linguistic-bound i.e., they have to rely on mere textual elements.

3.1.c) Additional materials:

              What kind of additional materials is available?

There are no systematic additional materials accompanying the book. Some guidelines about the dos and donts of the book have been very briefly explained in the introduction of the book.

3.2) The Micro Evaluation:

3.2.a) Unit grading:

                   How are the materials graded and sequenced?

In terms of sequencing, the units(extracts) have been organised in such a way that easier ones come first followed by more difficult ones. The first 20 extracts [belonging to Reading Comprehension (1)] are easier than the other 40 passages. This facility can be in terms of the lexical and grammatical complexity of the text.

3.2.b) Skills (Reading comprehension):

                   What type of reading materials as well as strategies does the book contain?

Three categories of passage extracts are included in the book: short, medium, and lengthy. The first is for the Reading Comprehension(1), the second for the Reading Comprehension(2), and the last one for the Reading Comprehension(3). The strategies advocated throughout the book comprise reading for gist or detail. This has been tested via numerous multiple-choice items.

3.2.c) Other skills:

Because this book encourages one of the receptive skills (reading), it does not include any other skill such as the productive ones or even the listening one as a receptive skill.

3.2.d) Lexis and grammar:

                   What approach to lexis and grammar does the material take?

The book does not intend to teach grammar or lexis as discrete components. However, the degree of lexical or grammatical difficulty has been taken into account throughout the passages. The approach seems to be an a priori one in which the trainees are expected to progress after they have accomplished the course with the book.

3.2.e) Learners/teachers role:

                   What is the learners/teachers role?

The learners are expected to get themselves involved actively in the activities the book has provided. They should work on the extracts lexically as well as grammatically to be able to carry out to have perfect skimming or scanning over the text contents. The teachers role cannot be ignored here although the book seems to have been presented on a learner-centred basis. The teacher is the one who makes all the significant decisions. But the attempt is to induce more learner-centeredness to learning. The teacher can act as a facilitator, guide, or teacher to teach some basics.

                   Does the material allow for differentiation?

The answer may be yes. The reason is that despite other books on reading comprehension, this book lacks redundancy: some books contain a pile of repeated exercises or activities, for instance, they test comprehension via more than one type of exercise such as multiple-choice, true/false, or match items. This book has a rather uniform structure in terms of its extracts and tests. Hence, it is a different material.

3.2.f) User-friendliness:

                   Is the book attractive?

The back cover of the book has a rather attractive designation: it is colourful containing information in Persian and English about the book specifications. The arrangement of the passages and the exercises is satisfactory.

                   Does the book have any auxiliary material?

The only auxiliary material of the book is a test bank that is available as the multiple-choice test at the end of each unit.

3.3) The compatibility of the book with the Iranian curriculum/syllabus:

Referring to the objectives of the Iranian curriculum, it can be concluded that the book under study is not compatible with all of the objectives: it is suitable for improving the trainees comprehension ability while it cannot increase their reading speed since there is no activity to focus on the skill. It helps the trainees enlarge their ability of perceiving words meaning from context, moreover, it is a good help for the students to guess meaning from context and the contextual clues. The material lacks the power to strengthen students comprehension ability of the implicit meaning in the text because there is no practice encouraging this ability: there is no word list with enough contextual background. Finally, the skimming and scanning skills may be practicable to some extent, however, because of the complexity level of the passages, the tasks take very long time, and it would be necessary to add some preliminaries to the material as a whole: a word list for each passage, a complete table of contents, a complete bibliography, some grammar exercises which elaborate the points in the passages, and finally, a number of pictures to be helpful in the process of getting meaning from context.

4. CONCLUSION:

This study shows us that the responsibility for constructing and preparing a new material is a prominent one. Once the basic criteria necessary for such a programme are not observed as they must be, the product which is in fact the material, may be unsatisfactorily designed and consequently useless. It is suggested that the draft of the material, particularly, when it to be a book, must be reviewed by a number of experts so that they can comment on the work to remove the possible deficiencies. The disadvantageous components of the content can by no means prevent the book from being used. The book in this study can certainly be evaluated via some criteria that have not been mentioned here. The evaluation format presented here is not claimed to be the unique one. Finally, while writing a resource such as a book, care must be taken to conside about the linguistic features related to the skill (here reading comprehension). The book in this study would have manifested itself more fruitfully if it had included preliminary instructions on the fundamentals of reading comprehension skill and the prerequisites before going into text extracts.

APPENDIX

The specification of the book in the present study:

Title: Reading Comprehension

For the University Students

Authors: Sh. Jahandar & M. Khodareza

First published: 2001

Published by: Daneshafarin Publications

ISBN: 964-7035-03-9

REFERENCES:

Alderson I., J. Charles Evaluating Second Language Education. 1992.C.U.P

Ansary H. & E. Babaii. Universal Characteristics of EFL/ESL. 2002.TESL

Chastain, K. Developing Second Language Skills. 1988 B.J.I.

Garinger D. Textbook Selection for the ESL Classroom. 2002. Eric Digest

Harmer J. How to Teach English. 1998. Longman.

Jahandar Sh.&M. Khodareza Reading Comprehension 2001 Daneshafarin Publications.

Johnson R.K. The Second Language Curriculum. 1989. C.U.P.

Widdowson, H.G. Teaching Language as Communication 1978 O.U.P.