An evaluation of an English as Second Language (ESL) Pakistani college textbook: Meeting the needs of the Pakistani students

This study investigated how textbooks influence learning experiences and aims to highlight the pedagogical implications of the incorporation of materials as the backbone of language-teaching programme. It analysed and evaluated a prescribed textbook (“Prose and Heroes” – a Compulsory English course book for intermediate classes in Pakistan), and explored the degree it benefited and challenged both the learners and the teachers. The research project utilized both the teacher’s and the learner’s perspectives and was based on Dubin and Olshtain (1986) textbook evaluation model. The main purpose of this study was to arrive at conclusions that would contribute to the improvement of the English language programmes in Pakistani colleges. The results of this study show that the relevant textbook does not cater to the needs of the learners. The data also implied that more attention needs to be given to the English language textbooks being utilized in language programmes.


INTRODUCTION
It is essential to determine whether the current English textbooks are effective in meeting the requirements of English language teaching and the students of English. This study was designed to assess the congruity between one prescribed textbook with the needs of the users in Pakistan. It was based on the premise that the Pakistani official English language programmes generally utilize substandard textbooks which are not facilitating the acquisition of the target language among the students. The research project focused on one compulsory English course book prescribed in the intermediate programme (classes XI and XII) in Lahore (the capital of the province of Punjab). This paper argues that the selected textbook "Prose and Heroes" may be confirming to the requirements of the publishers, authors and administrators, but it is definitely not meeting the needs of learners and teachers.

"Needs analysis"
The term "needs" in the context of education and curriculum development implies requirements of the students or any other relevant parties. "Needs analysis" refers to *Corresponding author. E-mail: elt@kinnaird.edu.pk Author agree that this article remain permanently open access under the terms of the Creative Commons Attribution License 4.0 International License the process of highlighting these requirements and evaluating the extent the proposed objectives, techniques and materials are compatible with the needs. Munby (1978) introduced the concept of "needs analysis" in the field of English for specific purposes (ESP). He primarily focused on the learners' language learning objectives and their future language requirements. This concept was further elaborated by Hutchinson and Waters (1987); they classified needs into the learners' requirements in relation to the target situations ("target needs") and their learning preferences ("learning needs"). According to the contemporary viewpoint, needs analysis should not be only limited to the "target needs" and "learning needs" (collectively termed "objective needs"), but it should also take into account the students' "subjective needs" which include their interests, desires and tastes (Nunan 1988). Language syllabi and materials should be based on the needs of the target learners (Graves, 2000;Bodegas, 2007). Graves assumes that the aim of "needs analysis" is to highlight the students' current backgrounds, preferences and attitudes as well as their future requirements.

Textbooks and language Learning
Most teachers of English use textbooks. In general, EFL/ESL textbooks generate a range of reactions. Responses often fluctuate between these two extremes. On one hand, these books are judged to be valid, useful and labour-saving tools. On the other hand, it is held that textbooks are "masses of rubbish skillfully marketed" (Brumfit, 1980). Sheldon (1988) identified three main reasons that the textbooks are so heavily utilized. First of all, teachers find developing their own classroom materials an extremely difficult, arduous practice. Secondly, teachers have limited time in which to develop new materials due to the nature of their profession. Thirdly, external pressures restrict many teachers. Each of these reasons present an accurate analysis of the strains experienced by teachers, and using course books is one of the most efficient and readily available ways in which to relieve some of these pressures. They lessen preparation time, and provide ready-made activities and concrete samples of classroom progress through which external stakeholders can be satisfied. According to Sheldon, textbooks represent for both teachers and students the visible heart of any ELT programme. However, there are other less positive reasons for textbook use. Often, rather than choosing textbooks that fulfill the goals of the curriculum, "an approved textbook may easily become the curriculum in the classroom" (Lamie, 1999). This is an unfortunate scenario since learners' needs are subjugated in favour of the limited possibilities of the text. As Cunningworth (1984) asserts, textbooks are good servants but poor masters. The teacher should use the textbook actively, which means that the teacher should formulate objectives with the needs of the learners Aftab et al. 145 in mind and then seek out published materials which will satisfy those objectives.

Materials evaluation
No textbook is perfect. Any textbook should be used judiciously since it cannot cater equally to the requirements of every classroom setting (Williams, 1983, p. 251). Therefore, teachers should have the option of evaluating and assigning materials based on their own specific needs in their own specific teaching situation. However, choosing or evaluating a course textbook is a daunting and specialized procedure, and in many contexts this is compounded by considerable professional, financial and political pressures. Moreover, it is vital that all involved individuals (from policy makers and administrators to teachers) should strive to effectively match textbooks with the identified needs of students.
It has been observed that a course book is a staple in every classroom, yet relatively little research and investigation has been conducted on materials evaluation. It is true that some linguists (Hutchinson and Waters, 1987;Sheldon, 1988;Skierso, 1991;Tomlinson, 1998Tomlinson, , 2003Tomlinson, , 2008McGrath, 2002) have suggested ways to help the administrators and teachers become more systematic in their approach, and have often offered guidelines and checklists based on broad ranged criteria. However, in many educational scenarios, the discussed principles are in actual fact not applied resulting in ad hoc materials evaluation (Tomlinson, 2003) based on simplistic criteria like "appropriateness of grammar presentation" and "popularity", ignoring the perspectives of the main stakeholders, that is the students, whose views on the materials they have been using need to be taken seriously" (McGrath, 2002, p. 199). Many experts advocate a very detailed examination of the textbook's language content, which has led to the production of extensive evaluation checklists (like those presented by Cunningsworth, 1984 andSheldon, 1988). Other checklists focus on the process of learning and highlight the range of cognitive skills needed by students to complete the activities (Skierso, 1991). This approach has been further extended by Littlejohn and Windeatt (1989) who seek to "look beyond the goals of language learning itself" (p.174) and therefore, include issues such as "learners' perceptions of knowledge" (p.174), "language learning and roles" (p.174), learners' world view and general knowledge as well as their affective and cognitive development.
As revealed by this review, experts demand a great deal from materials, although their beliefs may not always reflect the view or situation of the classroom. However, an awareness of these issues is significant for enhancing one's ability to evaluate and chose the most relevant textbook.

The Dubin and Olshtain Model: The framework of the study
A very comprehensive matrix for textbook evaluation has been presented by Dubin and Olshtain (1986). It is based on the following set of questions which also constituted the research questions of this study: By whom and where were the materials developed? It is important to identify whether the materials were developed by a team of experts who are familiar with a particular system and student population, or were they produced for the international market which is at best concerned with the broadest possible definition of the target population.
Are the materials compatible with the syllabus? Compatibility should be evidenced for all points specified within the syllabus. The procedures, techniques and presentation of items must be in harmony with the specifications given in the syllabus.
Do most of the materials provide alternatives for teachers and learners? Alternatives may be provided in terms of learner tasks, learning styles, presentation techniques, expected outcomes and so on. This is a significant feature of effective materials since not all types of learning routes are suitable for all learners. Therefore, ideally textbooks should enable experienced teachers and autonomous learners to develop their own alternatives according to their needs and personal preferences.
Which language skills do the materials cover? Materials often reflect the developer's preference for some language skills at the expense of others. If this is compatible with overall goals, then this would lead to an effective realization of the syllabus. However, if there is lack of integration, which is independent of specified goals, then this might be a serious drawback of the textbook.
How authentic are the text types included in the materials? If a textbook contains only rewritten, watered down stories that were adapted from a particular book, students using that material might never have the opportunity of encountering authentic texts. Furthermore, variety of text types might be significant in exposing learners to genres they will most probably encounter beyond the course.
How do learners and teachers who have used the materials feel about them? It is necessary to gather subjective information in order to gain insight into how teachable and learnable the materials really are. Evaluation by learners and teachers is a very integral part of this study.
While evaluating course books, it is important to accept the reality that most language classrooms will be using the materials to fulfill the goals of the programme. At the same time, the prescribed textbooks should be geared to help students meet their needs. Thus, systematic materials evaluation is necessary to provide insightful data regarding the effectiveness of the course book within the relevant educational context. The review of literature (discussed above) has revealed that there is no best model of textbook evaluation. Contributors have identified key elements which have been adopted into a model which was judged to be appropriate for this study limited to the evaluation of one Pakistani Compulsory English course book prescribed for the public and private sector intermediate students.

METHODOLOGY
The study was exploratory and descriptive in nature. The researcher used mixed methods approach since both quantitative and qualitative paradigms could be used to support each other. The quantitative approach provided a quick and effective assessment of the students' beliefs and opinions regarding learning English and the relevant textbook. The qualitative data highlighted an in-depth picture of the issues being explored. The Dubin and Olshtain model of materials evaluation (presented above) was adopted since it was deemed to be the most appropriate tool keeping in mind the context -second language materials scenario in Pakistani colleges. The selected framework was broad and focused and could provide opportunities for utilizing quantitative and qualitative tools to elicit comprehensive and significant findings.
Since the purpose was both to analyse needs and evaluate course books , three research tools were considered suitable for the research project -document analysis, questionnaires (for the students) and semi-structured interviews (with teachers). The Document Analysis should have focused on the syllabus and its specifications. However, the syllabus does not exist, and the textbook itself constitutes the syllabus. The book was examined thoroughly. First of all, the preface was studied in order to identify the specifications of the syllabus and then the procedures, texts and activities were assessed in terms of whether these were in harmony with the claimed objectives. The language skills covered by the textbook were also scrutinized. Finally the effectiveness of the book was gauged in terms of the used language, the provided exercises, and the adopted approach. This document analysis elicited both descriptive and insightful data illustrating how the textbook fared when evaluated according to the Dubin et al. model. The interviews were semi-structured, and the sequence and wording of the questions was the same for all respondents. The questions aimed to find out the teachers' opinions about the relevant textbook; the inter-views highlighted overall perspectives, the deficiencies in the book and how far the academic and social survival needs of the learners had been taken into account. The most surprising feature of the data obtained from the interviews was the degree of honesty and candour exhibited by the interviewees; initially it had been expected that the teachers would be more cautious in expressing their views.
Closed self-completion questionnaires were used for this study. The purpose was to illustrate the beliefs and perceptions of the students who were either studying the textbook or had already taken the intermediate examination. The first part of the questionnaire focused on the learners' preferences and attitudes towards English language and culture. The second part required the students to select a number on a 1 to 5 Likert scale illustrating what they think of the textbook, especially how far the book was compatible with their needs and culture. The sample consisted of 50 students and 4 teachers. The students belonged to affluent families. Their average age was 17 to 19 years and they had been studying English for the past 10 years. This sample was selected since they appeared to be more vocal. The teachers chosen were the ones easily accessible, experienced (17 to 25 years) and trained. The research was designed within practical constraints of time and money, and was thus confined to a small sample which was restricted to the evaluation of just one textbook. The convenience and purposive sampling techniques were adopted. Such a sampling procedure is common in small-scale studies, though this reduces the likelihood of generalization of the data (Nunan, 1992). Consequently, the findings of this research project cannot be applied broadly, but the aims were fulfilled keeping the exploratory nature of the study in mind.

Document analysis
The textbook prescribed for intermediate classes "Prose and Heroes" was examined in depth. The essays included in the book have been compiled by Dr. Sadiq, who has also written the Preface. The syllabus and learning objectives are discussed in the Preface of this textbook. Perusal of this document highlighted that the main aim of the book was to provide "suitable" texts for the students. However, the word 'suitable' is vague and no elaboration of this term is provided. The Preface reiterates that the material is "modern"; however, this assertion is questionable since many of the essays (specifically "Using the Scientific Method" and China's Way to Progress") are outdated. Similarly, another assumption namely that the learners will find the included texts interesting is also debatable because of the narrow and uninspiring choice of topics; the main themes are limited to scientific areas and the international heroes. The course book primarily focuses on reading skills, touches upon the writing skill, and totally ignores the oral skills and grammar. Generally, the students are required to read the provided essays and then respond factually to the given questions; most of the answers can be copied verbatim from the texts. With the exception of a few questions, these activities do not utilize any cognitive abilities (like evaluation, prediction and analysis). The compiled essays are authentic but, as pointed out earlier, obsolete and thus it can be assumed that the texts are of little value and interest as far as the learners are concerned.

Students' perspectives
The opinions of the learners were gathered through structured questionnaires. Most of the respondents (52 %) believed that their main aim of learning English was to be able to speak English fluently. In addition, speaking skill is the most preferred language skill of the majority. The learners use English beyond the classrooms, but mainly as far as the receptive skills (like reading English newspapers and magazines, watching English programmes, Aftab et al. 147 and listening to English music) are concerned. Speaking in English is predominately limited to classroom interactions with teachers and English writing tasks to simple activities like composing emails and greeting card messages. The cultural inclinations (assessed by the students' preferences to festivals) indicated a partiality towards the majority religious festival and the local celebrations (specifically Eid and Bassant). All the learners admitted that their teachers solely relied on textbooks in the classrooms. The majority of the students expressed poor opinion about the selected textbook (as detailed below). Most (68 % and 52 %) of the respondents felt that book neither utilized interesting topics nor did they enjoy reading the included prose. 74% of the learners believed that the texts do not focus on Pakistan, while 64% held that these essays primarily reflect the American/European culture. About 60% of the students affirmed that the textbook neither encouraged extensive reading nor helped improve their production skills. However, a relatively smaller percentage of the respondents (46% and 36% respectively) believed that the course book did not improve their listening and reading skills. Similarly, 46% did not find the activities interesting or challenging. At the same time, above 65% learners held that neither the book improved their overall English nor made them interested in the language. Surprisingly (considering the majority's negative comments about the various aspects), only half of the respondents expressed overall dissatisfaction with the course book.

The Teachers' viewpoints
Four teachers were interviewed in a semi-structured format. The teachers stated that the textbook includes British texts which have been compiled by a Pakistani. As such, the compiler should have been aware of the interests and needs of the Pakistani college students; however, the selected material and the themes reflect either a lack of awareness or disregard of the learners' concerns as also affirmed by majority of the respondent instructors. Moreover, the selected texts are not even culturally appropriate since they are outdated and the students can relate to only a few essays. Overall, the course book is not catering to the students' needs. Half of the respondents asserted that the textbook is not preparing the learners for institutions of higher education. Moreover, the materials do not facilitate the development of accuracy and fluency, language skills, creative and critical ability. Furthermore, once again half of the teachers affirmed that the course book is not compatible with the learners' interests. The instructors believed that the syllabus does not merit attention since the examinations are merely based on the subject matter of the texts which is obsolete and of no real value to the target students. In fact, the quality of the course book was not important; it was the examination system which was inherently flawed. According to the respondents, the textbook reflects two kinds of biases. First of all, it seems to cater to the science students since a large number of texts incorporate subject matter exclusively related to science. Moreover, the essay of only one female writer has been included; more importantly, all the five discussed heroes are males. Finally, most of the teachers believed that the textbook needed major overhauling; recommendations of improvement included incorporating modern essays and shifting the major focus towards functional English. One respondent so strongly disapproved of the book that she wanted it to be thrown away.

CONCLUSION
This study utilized triangulation of methodology and data. Thus, the findings were elicited from three different perspectives (that of the researcher(s), the relevant students and the teachers). However, the reached conclusions were similar. The textbook fails to meet the objectives as specified in the preface and is incompatible with the target learners' interests and needs (namely that of promoting fluency and enhancing language skills). The evaluation highlighted a diverse set of shortcomings which need urgent focus and subsequent remedies in order to improve the English language skills of the learners. It is essential that a new textbook is designed utilizing updated material which is of interest and value to a variety of learners. In addition, meaningful activities which provide opportunities for skills integration should be incorporated so that the course book can cater to overall English language development, rather than focusing on merely reading skills. Here it is important to assert that the intermediate programme is vital for Pakistani students since this course should ideally train them to gain admission to the technical higher educational institutions and cope with the highly specialized experience as required in these universities. Thus, the prescribed English textbooks should inculcate in learners the relevant skills needed to continue their education. This study involved a very small scale evaluation limited to a single textbook and a small sample of respondents. However, considering the illuminating nature of the data elicited through this research project, it is imperative that similar studies are undertaken on a larger scale, and that learners based in both rural and urban areas from all over Pakistan are incorporated in the research. At the same time, similar research programmes conducted in higher education contexts abroad can also contribute towards materials improvement in Pakistan and beyond.