Kingdom of Saudi Arabia Ministry of Higher Education King Saud University Deanship of Higher Studies Department of English Language
An Evaluation of the Sixth Grade English Language Textbook for Saudi Boys' Schools
By Abdulrahman Ali M. Alamri
[email protected]
Supervised by Dr. Dogan Bulut
Thesis Submitted to the Department of English Language in Partial Fulfillment of the Requirements for the Master Degree of Arts in Applied Linguistics at the College of Arts at King Saud University
June 2008
2
المملكة العربية السعودية وزارة التعليم العالية جامعة الملك سعود عمادة الدراسات العليا قسم اللغة اإلنجليزية و آدابھا
تقويم مقرر اللغة اإلنجليزية 1425/1424ھـ 2005/2004م للصف السادس االبتدائية بمداس البنين بالمملكة العربية السعودية إعداد الطالب عبد الرحمن بن علي بن محمد العمري قدمت ھذه الرسالة استكماالً لمتطلبات الحصول على درجة الماجستير في اللغة اإلنجليزية تخصص لغويات تطبيقية بقسم اللغة اإلنجليزية و آدابھا بكلية اآلداب بجامعة الملك سعود
إشراف الدكتور /دوان بولوت
2008/1429
3
i
Abstract
The study evaluates the quality of the sixth grade English language textbook for Saudi boys' schools which was introduced at the elementary stage by the Ministry of Education in 2004. This research project evaluates a new textbook that is considered to be the foundation stone in the English language program in Saudi Arabia. A survey
questionnaire was used in this study to elicit the perspectives of 93 English language teachers and 11 supervisors in Riyadh Educational Zone about the textbook in question. The questionnaire consisted of 64 grouped under 12 main categories: the general appearance, design and illustration, accompanying materials, objectives, topic appropriateness, learning components, socio-cultural contexts, skills development, teachability, flexibility, teaching methods, and practice and testing. The data was subjected to analysis through descriptive statistics. Three different types of computations were done: a per-item analysis, a per-category analysis, and an independent samples t-test to clarify the equality and variance of the responses the two groups of the population (teachers and supervisors). The findings were generally in favour of the textbook except the for the teaching methods and some other sub-items. Out of 64 items in the questionnaire, only 13 items had arithmetic means less than 2.50. The category that had the highest mean was the one on learning components, while the category that had the lowest mean was the one on teaching methods. The findings also revealed that there were no significant differences between the means of the two populations of the study (teachers and supervisors) except on the flexibility of the textbook and the different natures of their jobs might be the reason behind this difference.
ii The study concluded with recommendations and suggestions for the improvement of the textbook.
iii ملخص الدراسة
تھدف ھذه الرسالة إلى تقويم مقرر اللغة اإلنجليزية للصف السادس االبتدائي و الذي قررت وزارة التربية و التعليم تدريسه في المرحلة االبتدائية للمرة األولى منذ عقود .و تكمن أھمية ھذه الدراسة في كونھا تقوم كتابا محدثاً يعد حجر األساس لبرنامج تعليم اللغة اإلنجليزية بالمملكة كما أن ھذا الكتاب لم يسبق له أن قيم على حد علم الباحث. تھدف الرسالة إلى بيان نواحي القوة و الضعف في المقرر و معالجة األسئلة التالية: ً مرضيا؟ .1كيف كان المظھر العام في الكتاب ً مرضيا؟ .2كم كان تصميم والعرض في الكتاب .3ھل كانت المواد المصاحبة ذات صلة بالموضوع قيد المناقشة؟ .4ھل تم تحقيق أھداف المنھج والتي من أجلھا ُاستخدم الكتاب؟ .5ھل تم اختيار موضوعات الكتاب بعناية ,بحيeث أن كeل فكeرة مeن شeأنھا أن تeساعد علeى الفھeم الكامeل للموضوع؟ .6ھeeل تeeم اختيeeار العناصeeر اللغويeeة بعنايeeة ,بحيeeث أن كeeل فكeeرة مeeن شeeأنھا أن تeeساعد علeeى الفھeeم الكامeeل للموضوع؟ .7كم كانت محتويات كال من السياق االجتماعي والثقافي مرضية في الكتاب؟ .8إلى أي حد كانت مھارات تعليم اللغة مرضية في الكتاب؟ .9ھل الكتاب مناسباً من ناحية مناسبته للتدريس؟ .10ھل الكتاب مناسباً من ناحية مرونته؟ .11ھل الكتاب مناسباً من ناحية الطرق التدريسية؟ .12ھل االختبارات واألنشطة والتدريبات المقدمة في ھذا الكتاب فعالة؟ .13ماھي نقاط قوة الكتاب؟ .14بالنظر إلى ما تم تصنيفه ,ما ھي المحاورالتي يجب تطويرھا؟ .15ھل توجد فروق ذات دالات إحصائية بين استجابات كل من المعلمين و المشرفين؟
iv و قد استخدمت الدراسة استبينا ً مكونا ً من أربعة و ستين فقرةالستقراء آراء 93معلماً و 11مشرفا ً بمنطقة الرياض التعليمية .و بعد استخدام التحليل الكمي االحصائي و مقارنة نتائجه بين عينتي الدراسة كانت النتائج كما يلي: .1من بين 64فقرة ھي فقرات اداة البحث كانت ھنالك 13فقرة فقط أقل من المتوسط ) (2.5و ھو ما دل على مدى قبول المعلمين و المشرفين للمقرر الدراسي. .2كانت جميع المحاور االثني عشر متحققه في المنھج باستثناء المحور المتعلق بالطرق التدريسية المستخدمة في المنھج التدريسي حيث كان المتوسط الحسابي العام لفقرات ھذا المحور أقل من .2,5 .2المحور الذي حصل على أعلى متوسط حسابي ھو المحور المتعلق بعناصر التعلم بينما حصلت طرق التدريس المستخدمة على أقل متوسط حسابي. .3ال يوجد فروق ذات داللة احصائية بين استجابات عينتي الدراسة )المعلمين و المشرفين( حول محاور الدراسة اإلثني عشر ما عدا في ما يخص محور مرونة الكتاب .و قد يعزى ھذا الفرق اإلحصائي بين استجابات العينتين حول ھذا المحور إلى اختالف طبيعة العمل الذي تقوم به كل عينة. و قد اختتمت الدراسة بموجز لھا و بعدد من التوصيات و المقترحات لتطوير الكتاب.
v
TABLE OF CONTENTS Abstract
i
Table of Contents
v
List of Tables
viii
Acknowledgement
ix
Dedication
x
Chapter One Topic Introduction and Problem 1.1
Introduction
1
1.2
Significance of the Study
2
1.3
Statement of the Problem
4
1.4
Purpose of the Study
5
1.5
Limitations of the Study
6
1.6
Definition of Terms
7
Chapter Two Review of Literature 2.1
Introduction
Teaching English as a Foreign Language in Saudi Arabia 10
2.2 2.3
2.4
10
Materials Evaluation
14
2.3.1 Defining Materials Evaluation
14
2.3.2 Arguments for and against using textbooks
15
2.3.3 Theoretical Studies
19
2.3.4 Empirical Studies
32
Conclusion
42
vi Chapter Three Methods and Procedures 3.1
Introduction
43
3.2
Participants
44
3.3
Research Method
45
3.4
Research Instrument
46
3.5
Reliability and Validity of the Research Instrument
47
3.6
Data Analysis Methods
49
3.7
Conclusion
50 Chapter Four Data Analysis and Results
4.1
Introduction
52
4.2
Results on Per-Statement Analysis
52
4.2.1 General Appearance
52
4.2.2 Design and Illustration
55
4.2.3 Accompanying Materials
58
4.2.4 Objectives
61
4.2.5 Topic Appropriateness
65
4.2.6 Language Components
67
4.2.7 Socio-Cultural Context
69
4. 2.8 Skills Development
72
4.2.9 Teachability
75
4.2.10 Flexibility
77
4.2.11 Teaching Methods
80
4.2.12 Practice and Testing
82
4.3
Results on Per-Category Analysis
85
4.4
Equality and Variance of the Means of the Two Populations
87
4.5
Conclusion
89
vii Chapter Five Summary, Conclusion, Recommendation and Suggestions
5.1
Introduction
94
5.2
Summary
94
5.3
Conclusion
95
5.4
Recommendations
97
5.5
Suggestions for Further Studies
98
References
99
Appendix A
104
Appendix B
108
viii
Lists of Tables
Table
Page
1. Study population
45
2. Cronbach’s alpha values of questionnaire categories
48
3. General Appearance
53
4. Design and illustration
56
5. Accompanying materials
59
6. Objectives
62
7. Topic Appropriateness
65
8. Learning component
68
9. Socio-cultural contexts
70
10. Skills Development
73
11. Teachability
76
12. Flexibility
78
13. Teaching methods
80
14. Practice and testing
83
15. Arithmetic means and standard deviations of the main part of evaluative
86
criteria 16. Equality of Variance of the means of teachers and supervisors.
88
17. Statements with the lowest means
90
18. Statement with the highest standard deviations
91
19. Statements with the least standard deviations.
92
ix
Acknowledgements
First of all, I thank God the all mighty for giving me the blessings and the strength to do this work. Secondly, I would like to express my grateful thanks to my supervisor Dr. Dogan Bulut for his effort, advice, guidance and care. I will be always in debt to him for his unlimited care and overwhelming kindness. My thanks are extended to the members of the committee Prof. Mahmoud Saleh and Dr. Ibrahim Ali Haji-Hassan. My deepest respect goes to Prof. Abdulrahman Al-Abdan, Prof. Muhammad Ziyad Kebbe, Dr. Sulaiman Al-Ma'arik and special thanks to Dr. Odah Al-Johani, the chairman of the English Language Department. My deepest respect and gratitude also go to my friends who extended their hands for help during the data collection period. I would like also thank Khalid Al-Jadow and Majed Al-Qahtani for their assistance and support. For their proofreading, appreciation also goes to Timothy Moloney, Rollo Mallard and Khalid Al-Mitairi. My thanks go to my father, mother, brothers, sisters, wife and children for their patience, support and prayers and special thanks go to my uncle: Major General Hasan Alamri for his unlimited support.
x
An Evaluation of the Sixth Grade English Language Textbook for Saudi Boys' Schools
MA Thesis by Abdulrahman Alamri dedicated
to my uncle
Major General. Hassan Bin Saeed Alamri
1
CHAPTER ONE Introduction 1. Introduction One of the general principles that govern education in the Kingdom of Saudi Arabia is that the individual citizen has the duty of the pursuit of learning. The government in turn has the duty of providing learning for its citizens. It is also the principle of the Kingdom that Arabic is the language of education in all of its stages. The Kingdom of Saudi Arabia, however, does not discount the fact that there is also a need to be open to the possibilities of communication outside the Kingdom and hence the importance of being able to learn the international language which is English. That is why one of the goals of the education system in the Kingdom is to provide students with proficiency in English as a way of acquiring knowledge in the fields of sciences, arts and new inventions, and of transferring knowledge and the sciences to other communities, in an effort to contribute to the spread of the faith of Islam and service to humanity (Educational Policy of Saudi Arabia, 1970). Nevertheless, the main reason for embarking on this type of study lies in the fact that the contents of English textbooks or textbooks in general have a significant association with the learning of the students. The content of English textbooks must not contain errors because these errors once imbibed by the student as the correct one will have adverse effects on his/her learning. There can be no doubt that evaluating the official textbooks for English in Saudi Arabia is a task of great importance to the future success of the program. The decision of which textbook to use is a decision that will affect an entire generation of Saudi Arabian children. By properly scrutinizing the text intended for use, it is possible to make sure that the educational needs of the students are being met appropriately. In addition, English language skills will be of inestimable value in the global business environment; so in order for the children of Saudi Arabia to be progressive and
2 competitive in the world market; it is necessary for them to be educated in the appropriate skills, with the appropriate tools. One of those tools is the proper textbook for teaching English. The evaluation of a textbook however may seem to be a tedious task. Since a relatively new English textbook is being utilized by students in the sixth grade, it is important that it passes through close scrutiny as to its physical appearance, design and illustration, objectives, content, flexibility, teachability, teaching methods and practice and testing. This is the only way that parents, teachers and students will be assured of reliable and good learning. The reasons above motivated the researcher to evaluate the English language textbook studied at the sixth grade elementary schools in Saudi Arabia. The significance of this choice is that this textbook was published only recently and evaluating it will be an urgent need and of a great value. The evaluation can provide a reference for the subsequent revisions and improvement of certain aspects of this fundamental textbook and related material as well. Another reason behind the choice is that this book constitutes the building block for primary English language education and that this is where students make their first contact with a foreign language.
1.2 Significance of the Study There are different views concerning why we evaluate materials used for teaching English. Hutchinson and Waters (1987) see it as an analytical “matching process: matching needs to available solutions”. This, they suggested, can be done according to the sponsor’s needs through a needs analysis. While this perspective is important, it is incomplete since it leaves out the views of other interested parties, such as the teachers and the learners (ibid, p. 97).
3 Materials evaluation is an educational necessity because it shows how a textbook can be improved or justified. Teaching materials have a direct influence on the process of learning and teaching. Nunan (1988) states that: materials are, in fact, an essential element within the curriculum, and do more than simply lubricate the wheels of learning. At their best, they provide concrete models for desirable classroom practice. They act as curriculum models and at their very best they fulfill a teacher development role (Nunan, 1988, p. 98).
Weir and Roberts (1994) state two main reasons for evaluating teaching materials. First, it provides evidence “which can inform theoretical disputes about directions to be followed in language teaching or in teacher education” (p.11). Second, it is a tool to indicate the suitability of particular approaches or techniques under given conditions and whether they meet the claims made for them. The textbook proposed for evaluation is the textbook adopted for use in Saudi boys’ elementary schools for students in the sixth year. Since this is the introduction to English for Saudi students, it is important for the textbook to be engaging enough to excite the students' interest, but still to thoroughly present the fundamentals of English in an ageappropriate manner. This makes the evaluation of the textbook a matter of high importance. Palmer (1922) stated that "If we take care of the elementary stage, the advanced stages will take care of themselves" (p. 13). The study has both theoretical and practical contributions. It has a theoretical contribution in the sense that it is poised to contribute to the definition of the characteristics of a good teaching material, particularly for a very important subject. Since English is considered as today’s language of international commerce and transaction, the oil-rich Kingdom cannot afford to be left out while its neighbors and
4 countries around the world are improving their English literacy. This timely evaluation, along with others, may prove to be a valuable input to the Ministry of Education by serving as a possible guide for similar future textbook evaluations. Furthermore, the study also aims to provide a practical survey of how teachers can best evaluate the textbook in question. In actuality, this evaluation would benefit not only teachers and students, which is always in the forefront of every educational improvement, but also textbook designers and publishers and, ultimately, decision makers. The gains would come in the form of advice to classroom instructors on areas where a textbook can achieve its goals and on what parts would they need to use supplementary materials. For textbook designers and decision makers, the study would give them a feedback about the textbook and how it can be developed.
1.3 Statement of the problem The English language is the international language that is being considered nowadays. It is one of the dominant languages of the global market. Saudi students interested in engaging in international business will need English skills. Students who already have a working grasp of English will be able to benefit from many options for quality education like going to college in the United States, Australia, New Zeeland, or in Great Britain and will find the adjustment to modern Western culture much easier to handle. Saudi Arabia has long standing ties to the United States and Europe, so that it is likely for students to benefit from studying English regardless of career choice. Even Saudi children who grow up and never leave their home country are likely to encounter English-speaking people. The continuing trend of globalization is creating a need for multilingual individuals. Teaching English to the children of Saudi Arabia will definitely open up new opportunities for them in the future.
5 Therefore, the researcher finds it important to evaluate the English language textbook that is being used by sixth grade students because this will form the foundation of the student’s learning.
1.4 Purpose of the Study The purpose of this study is to evaluate the Sixth Grade English Language Textbook for Saudi Boys' Schools that was recently released for the first time. The title of the book is English for Saudi Arabia by Al-Mofarreh, et al and was published in 2004. The book is prescribed by the Ministry of Education of the Kingdom of Saudi Arabia to be taught at the sixth grade in all government elementary schools in the country. The researcher chose to evaluate this book for two reasons: first, it is the foundation stone in the English language program and second, it is a new book and evaluating it is an educational necessity. The evaluation deals with twelve categories. These are the general appearance, design and illustration, accompanying materials, objectives, topic contents, language contents, social and cultural contexts, language skills, teachability, flexibility, teaching methods, and practice and testing. More specifically, the study aims to answer the following research questions: 1.
How satisfactory is the general appearance of the book?
2.
How appropriate is the design and illustration of the book?
3.
Are the accompanying materials relevant to the subject matter being discussed?
4.
Are the objectives of the course for which the book is used met?
5.
Are the topic contents carefully selected so that each concept will help in the full understanding of the subject?
6 6.
Are the language contents carefully selected so that each concept will help in the full understanding of the subject?
7.
How far are the social and cultural contexts appropriate in the book?
8.
How much is the teaching of language skills satisfied in the book?
9.
Is the book suitable with regard to its teachability?
10.
Is the book suitable with regard to its flexibility?
11.
Is the book suitable with regard to its teaching methods?
12.
Are the tests, activities and exercises provided in the book effective?
13.
What are the categories perceived as strengths of the book?
14.
With respect to what category does the book need to improve on?
15.
Is there a statistically significant difference between the responses of teachers and supervisors with respect to the 12 main categories?
1.5 Limitations of the Study The study has two limitations. First, the study cannot be generalized to private schools as they are allowed to teach different textbooks. Second, the study was concerned with evaluating the sixth grade elementary school textbook, English for Saudi Arabia.
7 1.6 Definition of Terms The following are some terms that have been used in the entire study. It will be of help if these terms are clearly defined prior the body of the study.
Content gradation “The arrangement of syllabus content from easy to difficult” (Nunan, 1988, p. 158).
Curriculum “principles and procedures for the planning, implementation, evaluation and management of an educational program. Curriculum study embraces syllabus design (the selection and grading of content) and methodology (the selection of learning tasks and activities)” (Nunan, 1988, p. 158).
Evaluation Brown & Rogers (2002) define the term evaluation as “the process of seeking to establish the value of something for some purpose” (Brown & Rogers, 2002, p. 289).
Foreign language “A language which is not normally used for communication in a particular society” (Tomlinson, 1998, p. x).
Materials “Anything which is used to help to teach language learners. Materials can be in the form of a textbook, a workbook, a cassette, a CD-Rom, a video, a photocopied
8 handout, a paragraph written on a whiteboard: anything which presents or inform about the language being learned” (Tomlinson, 1998, p. xi).
Material evaluation “The systematic appraisal of the value of materials in relation to their objectives and to the objectives of the learners using them. Evaluation can be pre-use and therefore focused on predictions of potential value. It can be whilst-use and therefore focused on awareness and description of what the learners are actually doing whilst the materials are being used. And it can be post-use and therefore focused on analysis of what happened as a result of using the materials” (Tomlinson, 1998, p. xi).
Objective “ A statement of describing what learners will be able to do as a result of instruction” (Nunan, 1988, p. 158).
PPP “An approach to teaching language items follows a sequence of presentation of the item, practice of the item and then production of the item” (Tomlinson, 1998, p. xii).
Second language “The term is used to refer to a language which is not a mother tongue but which is used for certain communicative functions in a society” (Tomlinson, 1998, p. xii).
9 Syllabus “a specification of what is to be taught in a language programme and the order in which it is to be taught. A syllabus may contain all or any of the following: phonology, grammar, functions, notions, topics, themes, tasks” (Nunan, 1988, p. 159).
10 CHAPTER TWO Review of Related Literature
This chapter is allocated to review the literature related to two main topics. The first topic deals with teaching of English as a foreign language in Saudi Arabia, tackling the educational policy in Saudi Arabia, the educational policy for teaching English and the curriculum of teaching English at the elementary level. The second is allocated to review the literature related to textbook evaluation. It deals with a number of theoretical studies that contribute to the field of materials evaluation. Some of these are theoretical and the others are empirical. Tomlinson (2003) contends that "materials development is both a field of study and a practical undertaking" (p. 1). Since the literature on language teaching materials is vast, the researcher selects the most pertinent.
2.1 Teaching English as a Foreign Language in Saudi Arabia In 1927, the first primary and secondary schools were established in Saudi Arabia. In the same year, both English and French languages were introduced at the secondary level, but with no specific syllabus. In 1953, the intermediate level was introduced and a specific syllabus for teaching of English was established for both intermediate and secondary levels (Al-Subahi, 1988). This syllabus was imported from Egypt, but did not go with some of the Saudi learners’ customs and needs. Furthermore, other factors such as the poverty of the country and the high levels of illiteracy contributed to the slow progress in learning (Al-Sadan, 2000).
2.1.1 The educational policy of the Kingdom The conference sponsored by the government of Saudi Arabia in 1958 marked a new era in the history of educational organization in the country as regional
11 educational offices were established for the first time to supervise education in their local districts. In 1970, the basis of educational policy was established. It included the following statement: Students learn to understand in a correct and comprehensive manner. This determination involves that the Islamic creed is planted and spread, and that students are furnished with Muslim values, teachings and ideals. The student is to be equipped with the skills and knowledge which enable him to contribute constructively to the development of the Saudi Arabian society economically, socially and culturally having been fully prepared to become a useful member in the building of his community (Ministry of Education, 1970, Article 28). Four stages for the educational system in Saudi Arabia were established by the Ministry of Education: kindergarten, primary, intermediate and secondary. The students usually start their education at the age of six and spend six years in the primary stage, three years each for the intermediate and secondary stages (Khafaji, 2004).
2.1.2 The educational policy for teaching English The emphasis on the importance of learning English in Saudi schools is obvious in the Minister of Education’s address to the English language teachers written in Arabic on the second page of all elementary, intermediate and secondary course books. He encourages the teachers to do the following: To prepare our children to use English, which has become one of the most widely used languages in the world, for the resumption of the Muslims’ role in human civilization through gaining knowledge in Arts and Science written in this language. This can be done without promoting morals and customs
12 which are contradictory to our religious beliefs (Ministry of Education, General Directorate of Curricula, 2002).
The general objectives of teaching English in Saudi Arabia according to the new curriculum document published in 2001 are: Students should be able to: 1. develop their intellectual, personal and professional abilities. 2.
acquire basic language skills in order to communicate with the speakers of English Language.
3.
acquire the linguistic competence necessarily required in various life situations.
4.
acquire the linguistic competence required in different professions.
5.
develop their awareness of the importance of English as a means of international communication.
6.
develop positive attitudes towards learning English.
7.
develop the linguistic competence that enables them to be aware of the cultural, economical and social issues of their society in order to contribute in giving solutions.
8.
develop the linguistic competence that enables them, in the future, to present and explain the Islamic concepts and issues and participate in spreading Islam.
9.
develop the linguistic competence that enables them, in the future, to present the culture and civilization of their nation.
10. benefit from English –speaking nations, in order to enhance the concepts of international cooperation that develop understanding and respect of cultural differences among nations. 11. acquire the linguistic bases that enable them to participate in transferring the scientific and technological advances of other nations to their nation.
13 12. develop the linguistic basis that enables them to present and explain the Islamic concepts and issues and participate in the dissemination of them (Al-Hajailan, 2005).
The objectives listed above aim at enhancing the understanding of English-speaking cultures and bridging these cultures with the Islamic culture of Saudi Arabia. In addition, they aim at equipping Saudi students with the English skills they might need for academic and commercial purposes.
2.1.3 The curriculum of teaching English at the elementary level Al-Hajailan (2005) states "English was taught at the elementary stage from 1348 to 1361 H and was stopped for undeclared reasons"(p. 2). In 2004, English was introduced to the elementary stage again, but to sixth graders only. The number of periods was reduced to two periods a week. The general objectives of teaching English language for the elementary stage, as stated in the student's book, are listed below. By the end of the elementary stage and within the assigned structures for this stage, pupils should be able to: 1. learn the basics of the English language that would form the foundation of its mastery in the future. 2. use the basic structures of English sentences. 3. learn the core vocabulary assigned for this stage. 4. listen to and understand simple English language. 5. Express themselves orally using simple English language 6. read and understand simple written English language materials. 7. write simple guided sentences in English language.
14 8. appreciate the importance of English language as an international language of communication, for introducing Islam, the Islamic nation's culture and the cultural achievements of Muslims to other nations. 9. appreciate the importance of English language as an international language of communication to benefit from the achievements of other cultures in accordance with Islam (Al-Mofarreh, et al, 2005, p. A).
2.2 Materials Evaluation This part tackles four topics: defining materials evaluation, arguments for and against using textbooks, theoretical studies and empirical studies.
2.2.1 Defining Materials evaluation The term evaluation has been used to define a variety of processes in the field of applied linguistics. Lynch (1996) defined evaluation as “the systematic attempt to gather information in order to make judgements or decisions” (p. 2). Harmer (2001) sees a distinction between evaluation and assessment. He stated that “the assessment of a coursebook is an out-of-class judgement as to how well a new book will perform in class. Coursebook evaluation, on the other hand, is a judgement on how well a book has performed in fact” (p. 301). In addition to text materials McGrath (2002) contends that “materials could include realia: real objects such as a pencil, a chair or a bag and representations such as a drawing, a photograph of a person, house or scene” (p. 7)
As for materials evaluation as a field of study, Carter and Nunan defines it as “the process of measuring the value of learning materials. This can be predictive pre-use evaluation, ongoing whilst-use evaluation or retrospective post use evaluation (p. 223).
15 2.2.1 Arguments for and against using a textbook Throughout the history of second language acquisition research, many studies have discussed the advantages and disadvantages of using a textbook. These studies revealed that the use of textbooks has its proponents and opponents. Sheldon (1988) (as cited in Garinger, 2001) identified three main reasons for which textbooks are heavily utilized. These are: 1. developing their own classroom materials is an extremely difficult, arduous process for teachers 2. teachers have limited time in which to develop new materials due to the nature of their profession 3. external pressures restrict many teachers
Garinger (2001) adds a number of reasons for which teachers usually use textbooks. He states: Each of these ( the reasons identified by Sheldon, 1988) is an accurate analysis of the strains teachers feel and using a textbook is one of the most efficient and readily available ways in which to relieve some of these pressures. They lessen preparation time, provide ready-made activities, and provide concrete samples of classroom progress through which external stakeholders can be satisfied.
Cunningsworth (1995) states that coursebooks have important multiple roles in ELT. He summarizes these roles as follows: 1. a resource for presentation materials (spoken and written) 2. a resource of activities for learner practice and communicative interactions 3. a reference source for learners on grammar, vocabulary, pronunciation, etc. 4. a resource of simulation and ideas for classroom activities
16 5. a syllabus (where they reflect learning objectives which have already been determined 6. a resource for self-directed learning or self-access work 7. a support for less experienced teachers who have yet to gain in confidence (Cunningsworth, 1995, p.7).
Graves (2000) lists the following as some of the advantages of using a textbook: 1. It provides a syllabus for the course 2. It provides security for the students because they have a kind of road map of the course 3. It provides a set of visual, activities, readings, etc., and so saves the teacher time in finding or developing such materials 4. It provides teachers with a basis for assessing students' learning 5. It may include supporting materials (e.g., teachers guide, cassettes, worksheets, video) 6. It provides consistency within a program across a given level, if all teachers use the same textbook. If textbooks follow a sequence, it provides consistency between levels (Graves, 2000, p. 174).
Graves (2000) discusses the disadvantages of using a textbook as well. He identified a list of eight disadvantages: 1. The content or examples may not be relevant or appropriate to the group you are teaching. 2. The content may not be at the right level. 3. There may be too much focus on one or more aspects of language and not enough focus on others, or it may not include everything you want to
17 include. 4. There may not be the right mix of activities (too much of x, too little of y). 5. The sequence is lockstep. 6. The activities, readings, visuals, etc. may be boring. 7. The material may go out of date. 8. The timetable for completing the textbook or parts of it may be unrealistic. (Graves, 2000, pp. 174-5)
Tomlinson (2001) states: Proponents of the coursebook argue that it is the most convenient form of presenting materials, it helps to achieve consistency and continuation, it gives learners a sense of system, cohesion and progress and it helps teachers prepare and learners revise Opponents counter that a coursebook is inevitably superficial and reductionist, in its coverage of language points and in its provision of language experience, it cannot cater for the diverse needs of its users, it imposes the uniformity of syllabus and approach, and it removes initiative and power from teachers (p.67).
McGrath (2002) states that a textbook is important because it sets the direction, content, and to a certain extent how the lesson is to be taught. Similarly he asserts it is significant to view the images that teachers have as this reflects their attitudes and beliefs toward textbooks which will impact on how teachers use textbooks.
McGrath (2006) surveyed the teachers’ and learners’ views for coursebooks. He compared learner’s and teacher’s attitudes towards textbooks. Using the minimalist form of data gathering, respondents of the aforementioned study, namely teachers and students
18 of Hong Kong, were asked to complete the stem of the statement “A coursebook is a…..” either through metaphor or simile. Data on metaphors for a coursebook of English language teachers and secondary level students were classified into categories according to the semantic relations among theses images and metaphors. Some examples of teachers metaphors are the following: (1) a coursebook is like a choker that can make you look good but can also make another feel suffocated; (2) a coursebook is a map (and as such can be deceptive in its apparent simplicity of direction and explanation); (3) a textbook is a thick wood, rich, you learn a lot, you see a lot [but] you get entangled, you get lost. (4) a coursebook is a smokescreen. (5) A textbook is like a pair of shoes. It takes time to choose one that you feel comfortable to wear for a long time. A bad pair will kill you, give you blisters. A good one will give you confidence to run, to jump, to fly high. Some examples of learner’s metaphors on the other hand are (1) a pair of glasses (which help me to see what the teacher is talking about more), (2) a beggar (no one likes to approach it), (3) a meteor (that makes you brilliant), (4) a glass of water (good for us and make us healthy), (5) like milk (it nurtures our brains), (6) toxic (like CO2, N2O, make me feel bored, sleepy, waste my time), (7) supermarket (you can get everything there). Results of McGrath’s study showed that teachers have a gap as to their orientation towards textbooks but what was more revealing according to McGrath were the nature and strength of the metaphors of the learners which reflect either positive or negative attitudes toward textbooks pointing to the significance learners attach to textbooks in terms of either its suitability or to the way teachers use them.
19 2.2.3 Theoretical studies There is a wealth of academic literature on the design, selection, adoption and evaluation of English language textbooks. Some have maintained that the design of the material should be primarily informed by L2 acquisition and teaching theory, while others have maintained the imperatives of implementing socio-culturally sensitive designs as would embrace and utilize pre-existing L1 linguistic characteristics and experiences. Through a review of selections from recent scholarship on the topic raised, contemporary pedagogical debates and academic opinions on the issues of L2 textbook design, selection, adoption and evaluation shall be presented.
Bruder (1978) suggests the use of a checklist of eight criteria: level, objectives, style, language, age, time, convictions, and competency. He claims that these criteria should be considered from the viewpoints of teachers and students.
Williams (1983) argues for the design and application of a more systemic evaluative approach. Recognising that the inherently complex nature of L2 teaching material evaluation has spawned various evaluation paradigms, Williams (1983) believes that the design and implementation of a single, comprehensive assessment framework would allow for both more consistent evaluation standards and impose optimal design criteria upon textbook designers. He proposes an evaluative scheme which assesses the effectiveness of the material from pedagogical, linguistic, general and technical perspectives. Textbooks should, therefore, be evaluated on their inclusion of general linguistic instructions and rules, their simplified presentation of linguistic techniques, such as grammatical and syntactical rules, their embrace of practice material, as in reading passages, which both teach language, as in vocabulary and grammar, and can function to guide students’ writing skills and, lastly, on their provision of phonetics
20 guides (William, 1983). The argued for, theoretically-informed, evaluation criteria would further function as a guide for the design of L2 textbooks.
Cunningsworth (1984) discussed the principles and criteria for course analysis. The criteria he suggested are discussed under the following headings: language content, selection and gradation of language items, presentation and practice of new language items, developing language skills and communicative abilities, supporting materials motivation and the learner, conclusions and overall evaluation. At the end of the book, he provided a checklist that should be used in course evaluation.
Tu'eimah (1985) proposed three different tools: a tool for language teaching textbook analysis, a tool for language teaching textbook evaluation, and a tool for measuring language teaching textbook readability. The analysis tool consists of fifteen categories. These are the book’s physical make up, the nature of the course, pre-preparation studies, the language of the book, method of teaching, language skills, grammar, vocabulary, linguistic drills, assessment and evaluation, cultural content, aids and activities, individualized learning, the teacher’s manual and others. The evaluation tool consists of eight categories: pre-preparation studies, language content, language skills, methods of teaching, drills and evaluation, aids, physical make up, and overall impression. Finally, Tu'eimah (1985) suggests the use of cloze tests for measuring readability. He provided an example of how to use cloze test in order to measure the readability of language teaching textbooks.
21 Sheldon (1988) discusses some of the common theoretical and practical problems in textbook design. He reviews literature on the subject of textbook evaluation and the previous evaluative proposals. Finally, he suggests an evaluation sheet which consists of seventeen elements: rationale, availability, user definition, layout and graphics, accessibility, linkage, selection and grading, physical characteristics, appropriacy, authenticity, cultural bias, educational validity, practice and revision, flexibility, guidance and overall value for money. He also provides several key questions to be asked about each feature.
Skierso (1991) suggests that after collecting some background information about the learner, the teacher, the course, and the institutional objectives, an evaluation of the material can be done under the following criteria: The textbook A. Bibliographical data B. Aims and goals C. Subject matter D. Vocabulary and structures E. Exercises and activities F. Layout and physical make up The teacher's manual A. General features B. Supplementary exercises C. Methodological and pedagogical guidance D. Linguistic background information
22 Cunningsworth (1995) states two main approaches to evaluate the coursebook. The first is the impressionistic overview, which entails a preliminary sift through a lot of new material from which to choose (1995:1). This approach has little relevance to my Saudi context, since materials are published and supplied exclusively by the Ministry of Education. The second approach is the in-depth evaluation. This is a complementary approach to the first one, but in this approach close examination is carried out as to “how specific items are dealt with, particularly those which relate to students’ learning needs, syllabus requirements, how different aspects of language are dealt with, etc.” (Cunningsworth 1995:2). He suggests the use of different checklists to cover all aspects of the coursebook. The checklists cover aspects such as: 1. Claims made by the coursebooks 2. Types of materials evaluation 3. Purposes of materials evaluation These aspects are covered within seven areas, which are: 1
Aims and approaches
2
Design and organization
3
Language content
4
Skills
5
Topic
6
Methodology
7
Teacher’s guide These areas are evaluated through a series of questions which try to cover all of
their aspects. They appear to be designed in order to fit a variety of contexts and would ultimately need to be adapted to fit specific contexts.
23 Hemsley (1997) identified three basic kinds of ELT materials evaluation. These are: an intuitive, impressionistic approach; a formal prior-to-use evaluation and; a process approach. The intuitive, impressionistic approach is based on the first impression, so it is done by going through the materials or by reading the publicity blurb. The second approach is a systematic and comprehensive one. It is usually based on a close examination of the material using careful checklists or questionnaires. The third one which is the 'process approach' involves predictive evaluation, choice of a material, followed by a post-use retrospective evaluation.
Ellis (1997) argues for the implementation of empirical evaluation methodologies. Proposing a two-part evaluative framework, Ellis (1997) maintains that L2 instruction material and textbooks should first be assessed according to predictive standards. The implication here is that L2 teachers would independently evaluate the material in accordance with one of the numerous established evaluation checklists. Following evaluation and selection, the textbook/material is adopted. Subsequent to the use of the material, the second step in the evaluation process begins. Referred to as the retrospective evaluation process, teachers/evaluators are called upon to either “impressionalistically” assess the material according to their own perspectives of its efficacy or by involving the student and asking for their input on the material used (Ellis, 1997, p. 37). This evaluative approach, therefore, embraces the input of key stakeholders, both teachers and students.
Chambers (1997) argues for the design and implementation of a standardized evaluative paradigm, within the context of single schools or districts. As he explains, L2 teaching is a complex and complicated undertaking and more often than not, individual instructors have their own preferences for teaching theories and styles. Accordingly, it is
24 probable that each has his/her own preferences regarding L2 teaching materials. Schools generally rise above evaluation and selection controversies by imposing the choice of teaching materials upon L2 instructors. According to Chambers (1997), this is a counterproductive as it engenders resistance among L2 teachers towards the instruction material. It is in light of the stated that Chambers (1997) recommends that L2 teachers to be involved in the design of the evaluation framework and the subsequent section of the material. Needless to say, this is a practical recommendation which constructively draws upon the teachers’ pool of L2 teaching experiences.
Littlejohn (1998) rejected Cunningsworth (1995) and Harmer’s (1991) checklists, claiming that they would “involve making general, impressionistic judgements on the materials” (p.192). Instead, he suggests a general framework for analyzing materials. This framework consists of three main concerns. First, the aspects of materials that we should examine from a pedagogical perspective can be grouped under two main points: publication and design. The term publication refers to "the physical aspect of materials and how they appear as a complete set or book" (Littlejohn, 1998: 193). On the other hand, design refers to "the thinking underlying the materials" (ibid: 193). The second concern deals with the key aspects of tasks and poses three questions. The first question is related to three things: process, focus and operation. Process refers to the actual role of learners in the classroom activities, focus refers to whether language form or meaning is focused. The second question deals with the participation in classroom activities highlighting the forms of activities and the different forms of classroom participation. The last question deals with "the content of the task" (ibid: 199) The third concern asserts the importance of evaluating the overall aims of the materials, content and tasks selection and sequencing, teachers' and learners' roles,
25 demands of learner's knowledge, effects, skills and abilities and the role of materials as a whole.
Hayes and Schrier (2000) contend that L2 English language instruction textbooks and teaching material should be evaluated on the basis of the explicated criteria. Specifically stated, when assessing the value and possible effectiveness of L2 teaching materials, it is incumbent upon evaluators to do so in light of the cultural-linguistic context of implementation. The reading exercise materials should draw from the students’ cultural background and should utilize symbols, places, names and events that are both culturally familiar and identifiable. The instruction material should be presented in terms that are not just familiar to the student but which allow him/her to constructively utilize their pre-existing linguistic tools and knowledge for the more effective absorption of the new.
Conceding to the imperatives of designing L2 textbooks towards greater culturallinguistic sensitivity, and the subsequent evaluation of the effectiveness of such material through an assessment of the extent to which it does so, Byrnes (2000) argues the psychological validity of this design and evaluation criteria. The primary obstacle to effective L2 learning is learner psychology. Expounding upon this, Spielmann and Radnofsky (2001) state that empirical studies have effectively established the fact that the L2 learning environment is primarily dominated by tension, emanating from learner anxiety. Accordingly, it is incumbent upon L2 textbook designers to address the referenced tension and anxiety, and break down the psychological barriers to L2 absorption, through the direct embrace of what is culturally and linguistically familiar to students and, the responsibility of L2 instruction material evaluators to assess the value of the material according to this criterion (Byrnes, 2001; Spielmann and Radnofsky, 2001).
26
Hedges (2000) suggests a two-stage process for evaluating the relevance of a book to a particular group of students. "The first stage is to assess the content of a book in relation to its professed aims. The second is to assess the book against the needs and context of the intended learners "(p. 357). She also lists some of key categories for evaluation. Each category includes a number of questions that vary according to learner factors, institutional setting, and sociocultural context. Her suggested checklist is divided into five main categories. These are: the view of language, the view of language learning, learner, the view of education and the environment of learning.
Concurring with the assertion that evaluative paradigms should be theoretically based and should function, not only as a framework for assessing the instructional value of L2 material but for the design of L2 material, Olivares-Cuhat (2001) proposes an alternative approach for the evaluation of second/foreign language writing textbooks at all educational levels. This approach has theoretical and pedagogical foundations. The theoretical foundation is based on cognitive theories of learning and writing, while the pedagogical foundation is based on a learner-centered teaching method known as Strategy-Based Instruction. This approach integrates the cognitive view of learning and writing with pedagogical foundation of Strategy-Based Instruction. More specifically, L2 writing textbooks are evaluated according to the following features: cognitive writing functions, learning strategies, linguistic features, and writing approaches. She claims that an appropriate L2 evaluative approach should be the result of integrating these features. Finally, an evaluation technique (a checklist) is presented. This checklist consists of six categories: composing processes, metacognitive strategies, cognitive strategies, social strategies, effective strategies, and strategy awareness.
27 Harmer (2001) argued that there is a difference between assessing and evaluating a coursebook and we should have a different procedure of each. In assessment, he followed a three-stage procedure for “…teachers to assess books on the basis of their own beliefs and their assessment of their students’ needs and circumstances”. Harmer’s evaluative method has three stages: 1. Teacher record This is a record in the form of a diary or a detailed checklist kept by the teacher to record how successful different lessons and activities have been. 2. Teacher discussion This would be a discussion that takes place among teachers who are using the same book to compare their experiences. This record can then be circulated amongst those teachers before coming to a unanimous conclusion as to whether the materials should be replaced or altered. 3. Student response There is a number of ways whereby students respond to the materials used, and how such responses are collected. One way is through oral feedback; another is through written responses to questions about the students’ favourite or least favourite lesson activities.
McGrath (2002) reviews many checklists and methods used in evaluating materials in recent literature. He distinguishes three main stages in evaluation: pre-use, inuse, and post-use evaluation. Focusing on the post-use evaluation for course books, which is the main concern for my context, he develops criteria for choosing a suitable method of evaluation. These criteria consider the following steps: 1
The first step is a consideration of relevant contextual factors and the gathering of information analysis of the material
28 2
The second step is a close evaluation using the checklist method which should require
careful tailoring “to the needs of the learners and the teaching context and the need for periodic updating recognized” (McGrath, 2002, p. 56). 3
The final step is the decision making phase which can be reached through a careful
and systematic approach that addresses the questions of validity and reliability. McGrath (2002) believes that a good checklist should fulfill the following functions: 1. provide comprehensive information of the sort that will facilitate evaluation 2. and comparison 3. while making as few demands on the evaluator as possible 4. lead for the selection of the materials which are appropriate for the context 5. but also contribute to the advancement of learning and teaching in that context. (p. 48).
In addition, he discusses the procedures for adapting and supplementing materials. He justifies the need to adapt materials by asserting that no material is perfect, and that a suitable adaptation can compensate “for the lack of match between course aims and learner needs on the one hand and what the textbook provides on the other” (p. 79). These adaptations can make the material more suitable for the circumstances in which it is being used. The material can be supplemented through the provision of additional exercises "borrowed from other published materials or specially written for the target learning group” (p. 102). Tomlinson (2003) contends that materials evaluation is a procedure that involves measuring some or all of the following: 1. the appeal of the materials 2. the credibility of the materials to learners, teachers, and administrators;
29 3. the validity of the materials; 4. the reliability of the materials; 5. the ability of the materials to interest the learners and the teachers; 6. The ability of the materials to motivate the learners; 7. the value of the materials in terms of short-term learning; 8. the value of the materials in terms of long-term learning; 9. the learners' perceptions of the value of the materials; 10. the teachers' perceptions of the value of the materials; 11. the assistance given to the teacher in terms of preparation, delivery and assessment; 12. the flexibility of the materials; 13. the contribution made by the materials to teacher development; and 14. the match with administrative requirements (p. 16).
He also defines three types of materials evaluation that vary in purpose, in formality, in personnel and in timing. These are pre-use evaluation, whilst-use evaluation and post-use evaluation. "Pre-use evaluation involves making decisions about the potential value of materials for their users". (p.23) It can be context-free, content-influenced or context dependent. This type of evaluation is often impressionistic in the sense that it consists of a quick judgement by a teacher or an institution and consequently some mistakes may appear during the actual use of the book. To avoid these mistakes or at least reduce them, Tomlinson (2003) acknowledges making an evaluation criterion-referenced. This can make it more principled, rigorous, systematic and reliable. Whilst-use evaluation "involves measuring the value of materials whilst using them or whilst observing them as being used" (p.24). This type of evaluation is more
30 reliable than pre-use evaluation in the sense that it makes measurements rather than prediction. It can measure short-term memory and what is happening in the learners' brains but cannot measure durable and effective learning. (Tomlinson, 2003, p.24) Post-use evaluation which is the most valuable as it can measure the actual effect of the materials on the users. He (ibid) states: In other words it can measure the actual outcome of the use of the materials and thus provide the data on which reliable decisions about the use, adaptation or replacement of the materials can be made (p. 25).
This type of evaluation can be used to measure both short-term and long-term effects. It can measure motivation, impact, achievability, instant learning and many other short-term effects. On the other hand, it is useful for measuring long term effect regarding its durable learning and application. The problem is that it is time and effort consuming activity.
Duff, Wong and Early (2002) concur, adding that the Canadian L2 experience supports the imperatives of both designing and evaluating English language teaching textbooks and material which immediately address the cultural-linguistic context of application. Studies carried out on two groups of fourth-grade French-speaking L2 learners over the span of one academic semester, validated this assertion. The group which was taught through a standard commercial English language instruction textbook attained a discernibly lower level of linguistic competency than the group which was given the alternate, cultural-linguistic sensitive, textbook (Duff, Wong and Early, 2002). The findings of the study indicated the importance of designing English language instruction material according to the stated criteria. Crandall and Basturkmen (2004) propose that L2 material be evaluated according to more pragmatic concerns. As the researchers contend, empirical and
31 experiential evidence suggests that even upon the attainment of a high degree of L2 competency, L2 students do not acquire conversational fluency and often misuse words, leading to their interpretation as “rude” (Crandall and Basturkmen, 2004, p. 38). In response to this particular concern, Crandall and Basturkmen (2004) assert that L2 teaching material should be evaluated according to their practical value and their worth assessed on the basis of whether or not they embrace the imperatives of teaching conversational/practical linguistic skills.
Acknowledging the use of short, teacher-designed, class-specific questionnaires to be used for course planning and textbooks development and innovation, Davies (2006) describes the benefits of using such questionnaires and prefers them rather than institution-wide surveys because the latter suffer from two main drawbacks. "First, the results are not always available to learners and teachers. Second, … they often reveal little about the teaching/learning context where it matters most – individual classrooms" (p. 4) and for these reasons he believes that the global data obtained from such surveys cannot easily be matched with the individual classrooms. On the other hand, classspecific questionnaire surveys focussing on the individual teacher and his or her classes can be more effective for reflecting local classroom events and contents for the following reasons:
1. they are more reliable and more relevant to the context in question 2. they contribute greatly to the aim of achieving more cohesive long-term course development 3. they are a tangible reminder for learners that they are regarded as valued participant in the classroom and that they can participate in shaping and developing the ongoing and future courses. 4. teacher self-development is a natural and desirable result of engagement in the process designing class-specific questionnaires. (Davies, 2006, p. 4, 5)
Davies (2006) provides the results of a practical study conducted by him as an example of the effectiveness of class-specific questionnaires. His Japanese university-
32 level learners participated in the study. He developed a questionnaire of 14 items divided into three categories. The first category asked questions about the extent to which the course was enjoyable, classroom atmosphere, the teacher, and about their likes and dislikes about the course. The second category dealt with the content of the coursebook, the teacher-made materials, the level of classroom tasks, and learning language skills. Finally, the last category asks questions about the learners' attitudes to studying English, their exposure to English outside the classroom, and about their participation in classroom activities. He (ibid) contends that learners' responses revealed wants and needs and led to significant changes in key elements of the course such as syllabus type, skills focused, task types, lecture-style teaching, and topics.
2.2.4 Empirical Studies Ali (1983) evaluated the English language textbook taught to the second grade boy's intermediate level in Saudi Arabia. The evaluation was based on two things: a textbook analysis checklist and questionnaires for English language teachers and English language supervisors. The questionnaire consists of the following categories: the introduction of the course, the course's subject matter, aids, exercises and activities, the teacher's manual, the course book’s layout and the physical make up. At the end of his study, he provided some recommendation and a specification of the modification that the course should undergo. Some of these recommendations are: 1.
The importance of using more colorful and attractive teaching aids.
2.
The pupil’s book should present more interesting and age-appropriate topics.
3.
The teacher’s manual should provide teachers with alternative ways for teaching every lesson.
33 Al-Jaser (1988) conducted a comparative study to evaluate the English language textbook taught at the first grade boy’s secondary schools and the textbook taught at the same grade in girl’s secondary schools in Saudi Arabia. She used a questionnaire of 73 items as a research instrument. English language teachers and supervisors participated in this study. The questionnaire was built around the following categories: 1. The introduction of the textbook A. The target audience B. The grounded theory of the textbook C. The objectives of the course D. Teacher's guidelines 2. The textbook's content A. The subject matter and concepts of the textbook B. The textbook's language and style C. Aids The findings of this study can be summarized as follows: 1. The textbooks that were evaluated did not adhere to all the criteria of a good textbook. 2. The degree of adherence to some criteria of a good textbook is not as strong as it should be. 3. The boys' book and the girls' book are similar to a large extent. The researcher explained the points of strength and weakness of the two books and recommended that these textbooks should be revised according to the students' gender, level, needs and further responsibilities.
Ereksoussy (1993) conducted a qualitative research in order to evaluate the textbook taught at that time to the first year at girls' intermediate schools in Saudi Arabia.
34 She used a 126-item checklist as criteria for her evaluation. The checklist consisted of six main categories: 1. The objectives 2. Content selection 3. Gradation and recycling 4. Presentation frames 5. Practice activities (drills and exercises) 6. Assessment models (model tests) Ereksoussy (1993) found that there was more correspondence than divergence. The percentage of adherence to the criteria of a good book was 82.38 %. The researcher provided some recommendations and suggestions for further improvement.
Al-Hijailan (1999) investigated the quality of the third-grade secondary school textbook in Saudi Arabian schools. Both quantitative and qualitative techniques were employed in this study. He used a questionnaire to collect the teachers’ and supervisors’ opinions about the textbook. Qualitative data were also collected from document analysis, content analysis, and interviews. One hundred twenty-eight teachers and supervisors responded to the study’s questionnaire and thirteen teachers, five supervisors, and two administrators were interviewed. The results of the study revealed that the book’s appearance, accompanying materials, academic content, cultural content, and evaluation techniques were satisfied in the textbook while the fulfillment of the national goals and the teaching methods were not. The results uncovered the areas of strengths, and weaknesses of the textbook. The areas of strengths included utilizing colors and teaching aids, having an informative teacher’s book, a good organization, a good introduction, and employing the learners’ culture in a way that made learning English easier, faster, and interesting. The areas of
35 weaknesses included having several books, goals were not known to teachers, the book’s cover was not attractive, sequence of lessons was not appropriate, reading passages did not have questions and were not recorded in cassettes, some grammar items were repeated, and composition was guided. In order to modify the textbook, the researcher suggested combining the pupil’s book, the workbook, and the writing book into one book, making the book’s cover more attractive by putting colorful pictures on it, reorganizing the sequence of the lessons, increasing and re-organizing grammatical items, recording the reading passages on cassettes, supporting reading passages with questions, putting model tests, and adding free compositions, translation and dictation. Finally, Al-Hijailan (1999) recommended that the textbook should be updated every five years. Tomlinson et al (2001) used a list of 133 course evaluation criteria to evaluate eight current adult courses published by four leading publishers in the UK. The textbooks evaluated were Language in Use and True to Life by Cambridge University Press, Cutting Edge and Wavelength by Pearson Longman, Inside Out and Reward by Macmillan Heinemann Press. The checklist used was divided under two main headings, overall criteria and coursebook specific criteria. The overall-course criteria consisted of eight categories, namely publisher's claims, flexibility, syllabus, pedagogic approach, topic contents, voice, instructions and teachability. The coursebook-specific criteria included four categories: appearance, design, illustration and reading texts. In addition, specific criteria for cassettes and CD ROM, teacher's book, workbook and video were used in this evaluative study. The study provided an evaluation for every individual coursebook and one list of positive trends and another of negative trends in current courses. The positive list
36 included seven trends and the negative one included sixteen trends. Some of these trends are as follows: •
Positive trends 1. the move towards stimulating more personal responses from the learner. 2. the attempts of many of the courses to encourage humour and fun. 3. the realism of many of the audio components of the courses.
•
Negative trends 1. focusing on speaking and listening and consequently neglecting reading and writing activities 2. The absence of controversial issues to stimulate thought, to provide opportunities for exchanges of views and to make the topic content meaningful to adult learners. 3. The neglect of activities which could make full use of resources of the mind by simulating multi-dimensional mental responses which are at the same time sensory, cognitive and affective. (Tomlinson, B, et al., 2001, pp. 97-8).
Atkins (2001) provided a practical study wherein he used McDonough and Shaw's (1993) model for course book evaluation as the criteria for his study. His checklist consisted of two stages. The first stage was to look at the external characteristics of the material which includes the book's introduction, table of contents, the claims made on the cover of the student's and teacher's books, the publisher's catalog and the internet homepage. The second stage is the internal analysis which included the presentation of the skills in the material, appropriateness of discourse, authenticity of listening topics, the appropriateness of speaking materials and the effectiveness of the teachers' guide. Atkins's most important findings were:
37 1. The book was still based on PPP and there was no attempt by the authors to incorporate consciousness-raising activities. 2. The teacher's guide should provide alternative ways for teaching the same lesson. 3. The book could be adapted and edited to provide a usable course book.
Khafaji (2004) evaluated the materials used to teach English to the secondary level in Saudi public high schools. He discussed the general educational policy of the Kingdom, and the specific policy for teaching English. Then, he discussed the influence of English in Saudi society within the cultural, social, and economic boundaries. He also presented an overview of the English language curriculum and of teacher training in Saudi Arabia. Furthermore, he reviewed different methods and frameworks for material evaluation and previous work in evaluating the learning/teaching context in Saudi schools. He used three different checklists in order to evaluate the textbook: Cunningsworth’s (1995) checklist to evaluate the textbook as a whole; Littlejohn’s (1998) framework to evaluate one unit of the textbook with reference to the sections related to this particular unit in the teacher’s guide; and a checklist adopted from both to evaluate the reading skill. The general outcome of this study suggested that: 1. The materials have failed to provide the students with an adequate source of interesting and academically purposed substance to achieve the aims and objectives set for the students’ learning process 2. The teacher’s guide’s total control over the teachers resulted in having materials taught with less diversity and flexibility which led to having teachers with limited teaching experiences
38 3. The Audio-Lingual approach, which is the underlying approach in teaching the materials, has been a factor in limiting the capabilities of accomplishing the aims and objectives of learning the language.
Khafaji recommended that there is a need to re-evaluate the learning/teaching context in public schools in Saudi Arabia and to adopt new materials that can reflect the needs of learning English in society taking the student’s and teachers’ needs into account.
Nitta and Gardner (2005) conducted an evaluative study to investigate the occurrence of conscious-raising (C-R) and practice tasks in nine contemporary ELT textbooks. They developed a framework of conscious-raising and practice task types, applied it to the aforementioned textbooks and then identified a number of current trends. All the nine intermediate-level, multi-course textbooks are available in bookshops in Britain and were published by major international publishing houses. The research focused on three grammatical items, namely present perfect, second conditional and reported speech. Five types of form-focused task types were identified. These are: grammar-conscious tasks, interpretation tasks, focused communication tasks, grammar exercises and grammar practice activities. The first three types of tasks were categorized as conscious-raising tasks (C-R tasks) and the last two types were categorized as practice tasks. After examining the textbooks, the researchers came up with the following observations: 1.
In the overall structure of the grammar section, every coursebook examined basically follows a Presentation - Practice approach.
2.
Both inductive and deductive approaches to grammar presentation are identified in the materials but more preference for inductive
39 presentation is observed. 3.
Both C-R and practice tasks are employed in the most materials. Only focused communication tasks do not recurrently appear.
4.
Shifting from interpretation-based C-R tasks to production-based practising tasks is a typical procedure in all the materials examined.
5.
Contemporary ELT textbooks usually juxtapose C-R tasks with practice tasks.
Johnson, K. et al (2006) conducted a collective case study to see how teachers of various lengths of teaching experience go about in analyzing a textbook in terms of approaches to evaluation as well as considerations in terms of book content. This case study further aimed to contribute to the literature as far as theorizing techniques of textbook evaluation (Johnson et al., 2006). The teachers who participated in this study were divided into three groups according to their teaching experiences (one year[T1], five years [T2], and twelve years[T3]). The technique of concurrent verbalization coupled with “think-aloud protocols” were used in this study to reveal inner thoughts of teachers regarding the effectiveness of an English textbook entitled Just Right by Harmer (2004) used by the intermediate level at the Lancaster University. Along with the main textbook came a teacher’s book, and a mini-grammar booklet containing transcripts of the listening activities. With the aid of video and audio tapes, the research group observed five categories of data, namely sequence of evaluation, teacher preferences, use of terminology, methodological concerns, and flexibility in usage.
Results showed that T1 was the least methodical of the three teachers and was the least familiar with the general layout of a textbook and proceeded from one reference
40 (textbook) to another (mini grammar booklet) then jumped back to the textbook and went to the teachers manual, not necessarily following a strict set of procedures for evaluation. In the case of T2 the method of evaluation proceeded by looking at the front and back covers of the materials then looking at the contents of the textbook then went through the teacher’s manual then compared it to the textbook. T2 again looked at the beginning of the textbook and went through the pages again before beginning the evaluation. T2’s focus of evaluation centered on what the teacher took as immediate needs of the student, such as topics, themes, vocabulary, and functional language. T3, covered wider concerns than T1 and T2 but focused on the linguistic rationale of the textbook such as the occurrence of language in chunks in a variety of activities. The depth of his evaluation far surpasses the two other evaluators and was not concerned with establishing categories but more on expressing his likes and dislikes on the book such as its features, activities, and options presented on how to use or go about the book. This framework which T3 followed also showed his flexibility because he was able to accommodate a wide array of expectations both from the students and the teachers as well. He was more concerned, however, with the needs of the novice teachers and long-term academic needs of students. Furthermore his evaluation included references to other textbooks he was familiar with. T3’s systematic approach to evaluating of a textbook and depth of insights may be a vital point in the sense that it sets degree of efficiency and effectiveness that may be needed due to the premium placed on the importance of textbooks in a language course (Johnson, K et al, 2006).
Reinder and Lewis (2006) reviewed some of the existing evaluative criteria for self-instructional materials in general education and language learning and based on this review, a new criteria was developed to guide self-access in the selection of materials.
41 The researchers reviewed six different checklists for evaluating self-access materials. These checklists were Gardner and Miller's (1999), Dickinson's (1987), Jone's (1993), Sheerin's (1989), Lockwood's (1998) and a checklist by the Language Resource Centre Project. The study was carried out in the English Language Self-Access Centre at the University of Auckland. For the researchers to be able to come up with their suggested criteria, they went through the following steps: 1. Self-Access Centre (SAC) facilitators were asked what they believed students looked for in materials; 2. a questionnaire was administered to twenty students in the SAC who were randomly selected. This questionnaire was constructed to ask about the features of good materials from their view points; and 3. a four-cycle trialing stage was employed to insure the validity and reliability of their new checklist (Reinder and Lewis, 2006, pp. 275-6).
The new checklist they came up with was divided into four main parts. The first asks questions on the selection of the resources, the second deals with the extent to which students can access the resources easily, the third focuses on the learning process and finally a part on learning how to learn. Thein (2006) conducted a study to evaluate the suitability and effectiveness of the textbooks used for teaching English to religious studies students at Myanmar Institute of Technology. Questionnaires, interviews, and classroom observations were used to collect data from both teachers and learners. The study aimed at investigating the extent to which teachers and learners expectations match the objectives of the program in developing the students' communicative skills and critical thinking.
42 The findings revealed that the textbooks used in the program were not suitable for the following reasons: 1. The textbooks did not contribute to achieve the needs and wants of both teachers and learners 2. They were not effective to promote students' critical thinking 3. They were not effective in improving the students' communicative skills for everyday life situations Thein (2006) suggested that the textbooks must be localized to meet the teachers' and learners' needs. He also emphasized the importance of adapting activities that encourage collaborative learning, natural and experiential interactions to help young people think critically.
6.4 Conclusion Despite the great importance of material evaluation, there appears to be very little empirical research compared with the theoretical one. Furthermore, the number of textbook evaluation studies in Saudi Arabia is still relatively very small. This study is different from other studies in the sense that it deals with a textbook that represents the students' first significant contact with English in the Kingdom’s educational ladder. In addition, to my knowledge, no one has ever evaluated this textbook.
43
CHAPTER THREE Methods and Procedures 3.1 Introduction This chapter describes the research methodology, the operational methods of the variables employed and the validity and reliability tests of the survey instrument. In addition, it presents the different methods of data analysis. This study aimed at evaluating the sixth grade English language textbook for Saudi boys’ schools. Materials evaluation is an applied linguistic activity through which teachers, supervisor, administrators and materials developers can "make judgments about the effect of the materials on the people using them" (Tomlinson, B. et al 2001, p. 15). It is also of an important value for the development and administration of language-learning programmes" (McGrath, 2001, p. 1) One of the main reasons for materials evaluation according to Cunningsworth (1995) is "to identify particular strengths and weaknesses in coursebooks already in use, so that optimum use can be made of their strong points, whilst their weaker areas can be strengthened through adoption or by substituting materials from other books" (p. 14). For the reasons above, a survey was conducted in this study to elicit the views of a representative number of English language teachers in Riyadh Educational Zone about the sixth grade English textbook for Saudi schools. The data and results were used to answer the following research questions: 1.
How satisfactory is the general appearance of the book?
2.
How appropriate is the design and illustration of the book?
3.
Are the accompanying materials relevant to the subject being discussed?
4.
Are the objectives of the course for which the book is used met?
5.
Are the topic contents carefully selected so that each concept will help in the full understanding of the subject?
44 6.
Are the language contents carefully selected so that each concept will help in the full understanding of the subject?
7.
How far are the social and cultural contexts appropriate in the book?
8.
Is the teaching of language skills satisfied in the book?
9.
Is the book suitable with regard to its teachability?
10.
Is the book suitable with regard to its flexibility?
11.
Is the book suitable with regard to its teaching methods?
12.
Are the tests, activities and exercises provided in the book effective?
13.
What are the categories perceived as strengths of the book?
14.
With respect to what category does the book need to improve on?
15.
Is there a statistically significant difference between the responses of teachers and supervisors with respect to the 12 main categories?
3.2 Participants The questionnaire targeted the whole population of teachers and supervisors in the Riyadh area. The total number of the study population is 127, 113 teachers and 14 supervisors. Questionnaires were sent to all the teachers and supervisors, but only 109 questionnaires returned, 98 from teachers and 11 from supervisors. Five of the questionnaires returned from teachers were excluded because they were not completed properly. Table 4 presents some information about the population of the study and the number and percentages of participants. It gives an idea about the questionnaires administered, returned, excluded and used in this study.
45 Table 1 Study population Questionnaires administered
Questionnaires returned
Questionnaires excluded
Questionnaires useful for the study
Supervisors
113 14
98 11
5 -
93 11
Total
127
109
5
104
Community Teachers
Percentage
82.3 78.5 81.8
3.3 Research Method A survey was done to gather the information needed. This is a method of data gathering from samples which are representative of a population. Brown & Rogers (2002) state “Surveys are any procedures used to gather and describe the characteristics, attitudes, views, opinions and so forth of students, teachers, administrators or any other people who are important to a study” (p. 142). While a survey is proven to be an effective tool, it is also expensive, time and effort exhaustive and practically useless if not carried out well. When conducting a survey, it is important that the researcher clearly understands the goals of the study so that the process will be properly directed. Furthermore, the researcher has to consider the feasibility of the research topic to be able to have realistic expectations from the respondents. Brown &Rogers (2002) state Writing good items is the first step in doing survey research. To do this, you need to be very clear about what you want to survey. You may first want to make a clear written statement of what it is you are setting out to accomplish in your survey and keep that in mind when you are actually writing items. Once you have written the survey, whether in the form of an interview or a questionnaire, you will also need to get feedback from colleagues on the quality of the items and/or to pilot the survey instrument with participants similar to the ones you will eventually be surveying (pp. 142-3).
46
Brown & Rogers (2002) present a number of negative items that should be avoided when writing a survey. This list of negative items include overly long items, ambiguous items, leading items, embarrassing items, items that respondent are incompetent to answer, double-barreled items and irrelevant items.
3.4 Research Instrument Survey questionnaires were used to evaluate textbooks used in the sixth year of elementary schools. This is one of the common tools used in conducting a survey. It can easily be administered and can gather sufficient information given that it is properly constructed. Surveys are usually conducted by using interviews or questionnaire or both. Interviews are done in a face-to-face format, on telephone or even in groups. Questionnaires, on the other hand, are administered in writing to individuals or members of groups. Interviews are usually made up of open-ended questions while questionnaires are predominantly made of closed-response items such as Likert scales, multiple-choice, yes-no, and ranking (Brown & Rogers, 2002). In this study, a uni-dimensional questionnaire was used as a research instrument (See appendix A). It was adopted from four sources: from a checklist by Tomlison, et al. (2001), a checklist by Ereksoussy (1993), a checklist by Thein (2006) and from a textbook evaluation model by Al-Hajailan (2003). At the beginning of the questionnaire, a set of demographic questions was used followed by 63 closed-ended questions grouped under twelve main categories namely: (a) the general appearance, (b) design and illustration, (c) accompanying materials, (d) objectives, (e) topic contents, (f) language contents, (g) social and cultural contexts, (h) language skills, (i) teachability, (j) flexibility, (k) teaching methods, and (l) practice and testing. The questionnaire was written in English and it contained 63 items based on the jurors' suggestions.
47 3.5 Reliability and Validity of the Research Instrument
Everitt (2002) defines validity as “the extent to which a measuring instrument is measuring what was intended” (p. 388). Key (1997) described the validity of the research instrument as a test to determine whether the questionnaire measures what it is supposed to measure. Mason and Bramble (as cited by Key, 1997) showed that there are three basic approaches to the validity of tests and these are content validity, construct validity and criterion-related validity. For this study, content validity was utilized since it seemed to be the approach that entails less resources and effort compared to others.
Content validity is an approach which measures the degree to which the questionnaire items represent the universe of interest. In order for content validity to be established in the questionnaire, the researcher must identify the overall content to be represented. To do this, questionnaire items must be chosen randomly from the content. These items are expected to be a good representative of all the information in each of the areas of the survey instrument. To identify the universe of content, it is usually suggested that a panel of experts in the field to be studied will be asked to identify a specific content area (Key, 1997). The questionnaire used in this study, which was adopted from a number of previous studies, was given to five linguists and specialists to judge its validity. Some items were modified and others were deleted according to the jurors' advice until the researcher came up with a final draft of 63 items that satisfied all of them. (See Appendix B) McGrath (2002) defines reliability as the accuracy and consistency of the measurement resulting from an assessment. It is “the extent to which the same measurements of individuals obtained under different conditions yield similar results” (Everitt, 2002, p. 321). Four methods can be used to assess the reliability of an
48 instrument : and these are retest method, alternative form method, split-halves method and internal consistency method (Key, 1997). For the determination of the reliability of this study’s questionnaire, the internal consistency method was used. This is because this method required neither the splitting of items into halves nor the multiple administrations of instruments. The most popular internal consistency reliability estimate is given by Cronbach’s alpha. Table 2 presents the Cronbach’s alpha values of all the twelve sections of the questionnaire. The values ranged from .881 to .900 which can be considered high reliability values. Table 2 Cronbach’s alpha values of questionnaire categories Questionnaire sections No. of items
Alpha
Section 1
7
.889
Section 2
7
.899
Section 3
4
.893
Section 4
10
.888
Section 5
4
.893
Section 6
4
.886
Section 7
4
897
Section 8
6
.881
Section 9
3
.891
Section 10
4
.888
Section 11
4
.900
Section 12
7
.886
49 3.6 Data Analysis Methods
In analyzing the survey data, the rating took the form of Likert Scales. Rating scales were numerically coded as 1 – strongly disagree, 2 – disagree, 3 –agree and 4 – strongly agree. This greatly facilitated the statistical analysis. The Statistical Package for the Social Sciences (SPSS) release 11 was used to analyze the survey data. The data was subjected to analysis through descriptive statistics. The mean and standard deviation of each question were calculated. The arithmetic means and the standard deviation of each statement were computed and the following three sets of computations were done:
3.6.1. Analysis on a Per-Question Level
The first sets of computations were those of the means and standard deviations of each and every question or statement. This provides an idea about the extent to which each characteristic is satisfied in the textbook. Consequently, a primary evaluation for the textbook was generalized.
3.6.2. Analysis on a Per-Category Level
The second part of the data analysis looked at and interpreted the data on a per-category level. The analysis was on the basis of what class or type of characteristics are indicated in the group of questions. The overall mean of each category was computed and the extent to which each category was satisfied in the textbook in question was identified.
3.6.3 The Equality of Means of Teachers' and Supervisors' Responses
As noted above, the participants of this study are the English language teachers and supervisors in Riyadh Education Zone. So, in order to compare the responses
50 of teachers and supervisors, a two tailed independent sample t-test was used. This is used to "test the equality of the means of two populations" (Everitt, 2002 p.366). 3.7 Conclusion Briefly, this chapter discussed the participants, research method, research instrument and data analysis method. A survey questionnaire of 63 items was used as a research instrument. 104 English language teachers and supervisors from Riyadh Educational Zone participated in the survey. The data was subjected to analysis through descriptive statistics. Three different types of computations were done: a per-item analysis, a per-category analysis, and a two-tailed independent samples t-test to clarify the equality and variance between the two groups of the population.
51
CHAPTER FOUR Results and Discussion 4.1 Introduction This chapter presents the analyses and results of the data collected and their interpretations. The data used in this study was primarily collected through the distribution of close ended questionnaires amongst Saudi Arabian English language teachers and supervisors, drawn from the available pool of sixth grade teachers and school supervisors. While the researcher distributed a total of 127 questionnaires, only 109 were returned, of which 5 were excluded for reliability and validity reasons. Accordingly, the final survey population totalled 104, 93 teachers and 11 supervisors. The results were statistically treated to identify the frequencies, percentages, mean scores and standard deviations on each of the statements of the questionnaire. Also, the total means and standard deviations of each of the main parts of the questionnaire were computed. In addition, the researcher interpreted the results of the study.
4.2 The Results of Per-Statement Analysis The first set of computations is of the frequencies, percentages, means and standard deviations of each and every statement in the questionnaire. This provides us with the strong and weak points of the textbook in question in a detailed manner.
4.2 .1 General Appearance As noted in the literature review, several researchers have highlighted the imperative of the appearance of a textbook. McDonough and Shaw (1993) emphasize the importance of external evaluation because it "offers a brief overview of the outside of the book" (p. 61.) They assert the need for a critical evaluation of the blurb, or the claims made on the cover of the students'/teacher's books, and of the introduction and the table of
52 contents. Tu'eimah (1985) contends that the evaluation of the general appearance, or the physical make-up as he terms it, is of great importance. The rationale behind the identification of first impressions as an important criterion for textbook evaluation lies in the idea that students, quite literally, tend to judge a book by its cover. If the cover is attractive and the text clearly organized so as to facilitate students’ negotiation through it, the likelihood of students being attracted to the textbook is high. Student attraction to a textbook incites interest in a subject and encourages them to study it, while an unattractive textbook has the opposite effect. Acknowledging the important of textbook attractiveness, the survey sought to evaluate the sixth grade text from that perspective. Informed by a variety of studies, the criteria for determining attractiveness were 1) cover; 2) font size; 3) orientation page; 4) detailed table of contents; 5) appropriate lesson headings; and 6) comprehensive bibliography.
Responses were in the form of a 4-point Likert scale, ranging from strong agreement to strong disagreement. The results are indicated in Table 3, below.
53 Table 3 General appearance S
The cover is informative and 1 attractive The font size and type used in the book 2 are appropriate for sixth graders. There is an informative 3 orientation page. The book has a complete and detailed 4 table of contents.
5
Strongly agree
agree
F
47
42
15
%
45.2%
40.4%
14.4%
F
81
23
%
77.9%
22.1%
F
68
36
%
65.4%
34.6%
F
69
35
%
66.3%
33.7%
F
50
54
%
48.1%
51.9%
SECTORS
Every lesson is given an appropriate title.
The book has an 6 appropriate glossary.
disagree
Strongly disagree
F
6
51
47
%
5.8%
49.0%
45.2%
Mean
Std. Deviatio n
3.30
0.71
3.77
0.41
3.65
0.47
3.66
0.47
3.48
0.50
1.60
0.59
On whether or not the cover of the book is informative and attractive, the results indicate that an extremely high percentage of the participants support the statement, 45.2% strongly agree and 40.4% agree, while, only 15 participants (14.4%) disagree with the statement. So, it can be said that the majority of the participants believe that the cover of the textbook in question is informative and attractive. The mean score is 3.30 and standard deviation is 0.71.
Table 3 also shows a complete consensus on the statement that the font size is appropriate. All the participants either strongly agree (77.9%) or agree (22.1%).
54 Accordingly, this statement scores the highest mean in the first section of the questionnaire, 3.77 at a standard deviation of 0.41. This indicates that the teachers and supervisors are completely satisfied with the font size and type of the book. Furthermore, all the participants support the statement that the book has an informative orientation page, over 65 % of the participants strongly agree and the others agree. The mean score of this statement reaches 3.65 at standard deviation 0.47 which indicates a complete consensus among participants on this statement (See table 3). As indicated in table 3, all the participants support the statement that the book has a complete and detailed table of contents, 66.3% strongly agree and 33.7% agree. The mean score is 3.66 at standard deviation 0.47, which indicates that the table of content of the book in question is complete and detailed. Concerning the issue of whether every lesson is given an appropriate title, about half of the participants strongly agree and the other half agree on the statement. The mean score reached 3.48 at a standard deviation of 0.50. Regarding the statement on whether or not the book has an appropriate glossary, the results show that none of the participants strongly agree with the statement above and only 6 (5.8%) participants agree. On the other hand, 94.2% of the participants either disagreed or strongly disagreed with the statement and accordingly it can be said that the majority of the English language teachers and supervisors in the Riyadh area are not satisfied with the glossary provided in the book in question. The mean score is 1.6 at a standard deviation of 0.59 (See table 3). As noted above, on five of the six evaluative criteria, the textbook was deemed appropriate. Only as pertains to glossary did a comparatively significant percentage of the participants find substantial room for improvement. It should be noted at this stage, however, that this is the perception of the textbook which teachers and supervisors, not
55 students, have. Nevertheless, their perception is uniquely important, not only because it is informed by experience but because they are the ones who will be ultimately responsible for teaching through this book and, indeed, of devising the ways and means by which to incite greater student interest and curiosity in the subject at hand.
4.2.2 Design and Illustration As important as is the appearance of the textbook, the research literature establishes design and illustration as even more so. McGrath (2002) considers the clear layout and presentation as one of the aspects that must be taken in consideration when evaluating a textbook. Ur (1996) states that “among the points of evaluating materials are the clarity of layout and print and the attractiveness of the illustrations” (p. 17). Tomlinson et al (2001) emphasize the importance of having "enough white space to provide relief and clarity" (p.89). If the textbook’s message is not clear, this means that teachers cannot use it to effectively teach their students and that students will confront tremendous difficulties as they try to use it for their own studies and homework. Clarity and appropriate design and illustration are integral to comprehensibility and hence, to effective learning and teaching. To evaluate the extent to which the design and illustration is satisfied in the textbook, seven criteria were used. These are depicted in tabular format below:
56 Table 4 Design and illustration S
SECTORS
Strongly agree
agree
disagree
Strongly disagree
Mean
Std. Deviatio n
2.95
0.68
3.39
0.49
2.93
0.54
2.50
0.80
1.81
0.97
1 There is a variety of design to achieve impact.
F
22
55
27
%
21.2%
52.9%
26.0%
2 There is enough white space to achieve clarity.
F
41
63
%
39.4%
60.6%
3 There is consistency in the use of headings, icons, labels, italics, etc.
F
12
73
19
%
11.5%
70.2%
18.3%
4 The illustrations are varied and attractive.
F %
20 19.2%
13 12.5%
71 68.3%
5 The illustrations stimulate students to be creative. 6 The illustrations are functional.
F %
12 11.5%
5 4.8%
39 37.5%
F %
9 8.7%
77 74.0%
18 17.3%
2.91
0.50
9
72
23
2.86
0.54
8.7%
69.2%
22.1%
7 The illustrations F facilitate students' visualizations without % imposing complete visual images.
48 46.2%
The results indicated that the majority of participants support the statement that there is a variety of design to achieve impact, 21.2 % strongly agree and 52.9 % agree. On the other hand, only 27 participants (26%) disagree. The mean score reached 2.95 at a standard deviation of 0.68 (see table 4) Concerning the issue of whether or not there is enough white space to achieve clarity, table 5 shows that there is a complete consensus on the statement. All the participants either strongly agree (39.4%) or agree (60.6%) on the statement. Therefore, this statement scored the highest mean in the second category of the questionnaire, 3.39 at a standard deviation of 0.49.
57 Furthermore, the majority of participants support the statement about whether or not there is consistency in the use of headings, icons, labels, italics, etc. Seventy-three participants (70.2 %) agree and 12 (11.5%) strongly agree, while only 19 participants (18.3%) disagreed. The mean score of this statement is 2.93 at standard deviation of 0.54. (See table 4) Regarding the illustrations used in the book in question, the majority of the participants, over 68%, disagree on the statement that "the illustrations are varied and attractive." Only about one third of the participants support the statement. The mean is 2.50 at a standard deviation of 0.80. The table also shows clearly that most of the participants perceive the book as not having illustrations that can stimulate students to be creative. Thirty-nine participants (37.5%) disagree on the statement and 48 participants (46.2%) strongly disagree. On the other hand, only 12 participants strongly agree and 5 agree. The arithmetic mean of this item is 1.81 at a standard deviation of 0.97. On whether or not the illustrations are functional, the results indicate that over 82% of the participants support this item, while 17.3% of them do not. That means that the majority of the participants believe that though the illustrations of the book were not varied and attractive, they are functional and can be useful. The mean score reached 2.91 and the standard deviation is 0.50 (see table 4). Finally, table 5 shows that the majority of the participants support the statement that the illustrations facilitate students' visualizations without imposing complete visual images. Over 78% of participants agree and 8.7% strongly agree. On the other hand, 22.1 disagree on the statement. The mean score of this item is 2.86 and the standard deviation is 0.54.
58 As indicated above, the results for this category were mixed. While not evaluating the textbook as unclear per se, the results quite evidently established it as not as clear as it should be. It is quite important to note that a significant percentage of the participants had negative opinions regarding the textbook illustrations. This is quite unfortunate considering that illustrations serve the purpose of clarifying complexities and of stimulating both interest and creativity. Sheldon (1988) suggests having "an optimum density and mix of text and graphical materials on each page" (p.243.) He also emphasizes the importance of having colorful and appealing artwork.
4.2.3
Accompanying Material The research literature tends towards the opinion that a textbook should not
be judged on the basis of that which it contains between its covers alone but that evaluation should expand to include the accompanying supplementary material, if any. For instance, McGrath (2002) asserts the availability of teacher’s book and video and audio components. Also, Tomlinson et al (2001) consider the use of CD-ROM and cassettes as one of the interesting and effective ways in the current generation of language teaching materials. In addition, Gower et al (1983) contend that the use of supplementary materials such as visuals, cassettes, video, and computers can make lessons more interesting and effective. For greater clarity and for the purposes of facilitating both students’ learning and teachers’ abilities to use the textbook in question as an effective instructional tool, the sixth grade English language textbook used in Saudi boys’ schools comes with accompanying materials for both students and teachers. These include additional teachers’ books and student's exercise books, flashcards and CD. So, the survey was directed towards evaluating the quality of these materials. The results of this particular criterion are outlined below.
59 Table 5 Accompanying Materials S
SECTORS
Strongly agree
agree
disagree
Strongly disagree
1 Cassettes that accompany the book are suitable.
F
67
37
%
64.4%
35.6%
2 The CD-ROM that accompanies the book is suitable.
F
49
55
%
47.1%
52.9%
3 The posters and flash cards that accompany the book are suitable.
F
25
57
22
%
24.0%
54.8%
21.2%
4 The teacher's book that accompanies the book is informative.
F
25
61
16
2
%
24.0%
58.7%
15.4%
1.9%
Mean
Std. Deviatio n
3.64
0.48
3.47
0.50
3.02
0.67
3.04
0.68
The results show that there is a complete agreement on the suitability of the cassettes that accompany the book. Over 64% of the participants strongly agree and 35.6% agree on the statement. None of the participants disagree or strongly disagree. So, it can be said that the cassettes accompanying the textbook are suitable. The mean score is 3.64 and the standard deviation is 0.48 (See table 5). Regarding the CD-ROM that accompanies the book, the table shows that there is a complete consensus on its suitability. Over 47% of the participants strongly agree on the statement and almost 53 participants agree. No negative responses are recorded. The arithmetic mean is 3.47 and the standard deviation is 0.50. Furthermore, there is a large percentage of the participants with the statement that the posters and flash cards that accompany the book are suitable. Almost 55% of the participants agree and 24% strongly agree while only 21.2% of the responses do not support the statement. The mean score is 3.02 and the standard deviation is 0.67 (See table 5).
60 On whether or not the teacher's book that accompanies the book is informative, the table shows that the results are in favor of the teacher’s book. Over 58% of them agree on the statement and 24% strongly agree. On the other hand, 15.4% of the participants disagree with the statement and 1.9% strongly disagree. The reason behind this conflict might be because the fact that the teacher’s book does not provide a variety of methods and technique for teaching the same lesson in order to give the teachers to chose the method or technique that suits their students.
In general, the participants tended towards the opinion that the accompanying material was appropriate for the textbook in question. Indeed, only a minority criticized posters, flashcards and the teacher's book. It is important to note here that this particular level of evaluation does not speak for the textbook itself but only for its accompanying material and, even then, the appropriateness of that material within the context of the textbook itself. Consequently, that the accompanying material was deemed appropriate does not mean that the textbook itself is appropriate or, at least, that the appropriateness of the textbook cannot be judged on the basis of the responses to this question-set. Ur (1996) states that "most language teaching coursebooks probably need supplementing to some extent" (p. 198.) She asserts the use of computers, videos, audios, posters, pictures, and games. McGrath (2002) asserts the availability of teacher’s book and video and audio components. Tomlinson et al (2001) consider the use of CD-ROM and cassettes as one of the interesting and effective ways in the current generation of language teaching materials. Gower et al (1983) contend that the use of supplementary materials such as visuals, cassettes, video and computers can make lessons more interesting and effective.
61 4.2.4 Objectives Needless to say, each textbook has an explicit set of objectives. The textbook is composed with those objectives in mind and its ultimate aim, or purpose, becomes the fulfillment of those objectives. McDonough and Shaw (1993) state that "…we may need to select materials that suit a particular syllabus or set of objectives that we have to work to" (p.70). As pertains to the textbook under study here, the objective is to teach Saudi Arabian sixth grade students English. This, of course, is easier said than done as fulfilling this particular objective is predicated on a very accurate understanding of the student group’s English proficiency level and must be informed by theory. Added to that, the textbook’s objectives must be explicit and the degree to which it satisfies them must be measurable. Within this context, ten criteria for evaluating the extent to which English language textbooks for non-native speakers satisfy this particular requirement were identified. These were included in the survey questionnaire and the results are tabulated in Table 6.
62 Table 6 Objectives Strongly agree
agree
F
38
66
%
36.5%
63.5%
F
41
63
%
39.4%
60.6%
F
24
62
18
%
23.1%
59.6%
The terminal objectives define the desired degree of mastery.
F %
16 15.4%
The terminal objectives meet the needs and wants of sixth graders.
F %
The developmental objectives are specified at the beginning of each lesson in the teacher's book.
Std. Deviation
3.36
0.48
3.39
0.49
17.3%
3.05
0.63
68 65.4%
20 19.2%
2.96
0.58
6 5.8%
66 63.5%
32 30.8%
2.75
0.55
F
32
72
%
30.8%
69.2%
3.30
0.46
The developmental objectives are clear and precise.
F %
52 50.0%
52 50.0%
3.50
0.50
The developmental objectives are measurable.
F %
53 51.0%
51 49.0%
3.50
0.50
The developmental objectives suit the level of sixth graders.
F
37
67
%
35.6%
64.4%
3.35
0.48
F
43
61
%
41.3%
58.7%
3.41
0.49
SECTORS
1
Generally, the book fulfills the general objectives of teaching English language in Saudi Arabia.
3
4
5
6
7 8
9
10
Strongly Disagree
Mean
S
2
Disagree
Generally, the book fulfills the general objectives of teaching English language for elementary stage. The terminal objectives specified explicitly in the pupil's book are based on some theoretical background.
The developmental objectives contribute to the attainment of terminal objectives.
63 Table 6 shows that there is a complete consensus among the participants on the statement that the book fulfils the general objectives of Teaching English language in Saudi Arabia. The majority, 63.5% of the participants, agree on the statement and 36.5% strongly agree. No one disagrees or strongly disagrees. The arithmetic mean is 3.36 at a standard deviation of 0.48. Furthermore, the results indicate again a complete consensus on the statement that the book fulfills the general objectives of teaching English language for elementary stage. Over 36% of the participants strongly agree on the statement. The rest of the participants agree on the statement. The mean reached 3.39 and the standard deviation is 0.49. Regarding the terminal objectives specified explicitly in the pupil's book, the results show the majority of the participants believe that they are based on some theoretical background, 59.6% agree and 23.1% strongly agree. On the other hand only 17.3% of the participants do not support the statement. The mean score is 3.05 at a standard deviation of 0.49 (See table 6). Concerning the issue of whether or not the terminal objectives define the desired degree of mastery, almost one fifth of the participants disagree on the statement. On the other hand, over 80% of the participants support the statement. The arithmetic mean is 2.96 at a standard deviation of 0.58 (See table 6). Furthermore, the results show that the majority of the participants support the statement that the terminal objectives meet the needs and wants of sixth graders, 5.8% strongly agree and 63.5% agree. On the other hand, about one third of the participants do not agree on the statement. The mean is 2.75 and the standard deviation is 0.55. These results indicate that the terminal objectives meet the needs and wants of the targeted students. With regard to the developmental objectives, there is a complete consensus among participants on that they are specified at the beginning of each lesson in the teacher's
64 book. Over 30% of the participants strongly agree and a little less than 70% agree The arithmetic mean reached 3.30 at a standard deviation of 0.46 (see table 7). On whether or not the developmental objectives are clear and precise, table 7 shows a complete agreement as half the participants strongly agree and the other half agree on the statement. The mean is 3.50 at a standard deviation of 0.50(see table 6). These results indicate that the developmental objectives are clear and precise. This is an important indication in the sense that precise objectives facilitate the teachers' job to achieve these adjective. Regarding the issue of whether or not the developmental objectives are measurable, the results show that a little more than half of the participants strongly agree and the rest of them agree. The results do not show any negative responses. The mean is 3.50 at a standard deviation of 0.50 (see table 6). On whether or not the developmental objectives suit the level of sixth graders, there is again a complete agreement on the statement. Over 35% of the participants strongly agree and almost 65% of the participants agree. The arithmetic mean is 3.35 and the standard deviation is 0.48 (see table 6). The table, finally, shows that 58.7% of the participants agree on the statement that the developmental objectives contribute to the attainment of terminal objectives while 41,3% strongly agree on the statement. The arithmetic mean is 3.41 and the standard deviation is 0.49 (see table 6). As indicated in the above outlined results, there is a general consensus that the textbook clearly outlines and satisfies its objectives. There is, however, some disagreement on one very important point. This is whether or not the textbook satisfies the needs and wants of Saudi Arabian sixth grade public school English language learners. The fact that close to a third of the participants opined that it did not detracts from the value of the textbook in question. It is important to remember here that the
65 participants are not only professionals in the field with a string academic grasp of the complexities of teaching English as a second language but they have years of experience behind them; experience in teaching English.
4.2.5 Topic Appropriateness With particular reference to English language textbooks, the issue of topic appropriateness emerges as one of predominant concern. This was explicitly stressed in many of the studies of the literature of this research (Sheldon,1988, Skierso 199, Cunningsworth 1995, Hemsley 1997, Littlejohn 1998). Bearing this in mind, the survey questionnaire evaluated the topic appropriateness of the textbook in question by including four criteria as indicated in Table 7. Table 7 Topic Appropriateness S
Strongly agree
agree
F
43
56
5
%
41.3%
53.8%
4.8%
F
37
21
38
8
% F
35.6% 92
20.2% 12
36.5%
7.7%
%
88.5%
11.5%
F % %
1 1.0%
8 7.7%
SECTORS
1 The topics of the book are varied and engaging to appeal to sixth graders with different interests and personalities. 2 The topics encourage students to express their own views. 3 The book avoids potentially embarrassing or disturbing topics. 4 The topics allow students to think critically.
disagree
49 47.1%
Strongly disagree
46 44.2%
Mean
Std. Deviatio n
3.36
0.57
2.83
1.00
3.88
0.32
1.65
0.66
In table 7, over half of the participants agree on the statement that the topics of the book are varied and engaging to appeal to sixth graders with different interests and personalities and Forty-three participants (41%) strongly agree. Only about 5% of the
66 total population disagree with the statement. The mean score is 3.63 at a standard deviation of 0.57. Furthermore, the results show that 35.6% of the participants strongly agree and 20.2% agree on the statement that the topics encourage students to express their own views. The results also show that 36.5% of the participants disagree and 7.7% strongly disagree with the statement. The arithmetic mean is 2.83 and the standard deviation is 1.00 (see table 7). Regarding the issue of whether or not the book avoids potentially embarrassing or disturbing topics, the results show that all the participants’ responses are in favor of the book as 88.5% of the participants strongly agree and the rest agree on the statement and no one disagrees or strongly disagrees. This means that there is complete agreement that the book does not include any embarrassing or disturbing topics. The mean score of this item is the highest in its category. It reached 3.88 at a standard deviation of 0.32 (see table 7). On the other hand, the above table shows that over 90% of the participants do not support the statement that the topics allow students to think critically. Forty nine participants disagree and 46 strongly disagree with the statement. Only 9 participants support the statement. The mean score is 1.65 and the standard deviation is 0.66.
As indicated above, the survey results indicated a generally positive perception regarding topic appropriateness. Importantly, the textbook was further perceived to include many topics which are varied and engaging to appeal to students with different interests and personality types. Nevertheless, and this is quite an important finding, supervisors and teachers largely tended towards the view that the textbook does not encourage critical thinking. This is very unfortunate as such topics can be very helpful for most students. Thein (2006) found that "most students enjoy discussing the topics that
67 stimulate them to generate their critical thinking and leave room for making judgments on others' opinions" (p.58).
4.2.6 Language Component Each subject is taught and learnt through its components. English is not taught as a whole but as a subject which is comprised of various components, all of which compliment and complete one another. As far as language classes are concerned, these components are grammar, vocabulary and phonetics. Evaluating these components is essential for most scholars as observed in the literature of this study (Sheldon,1988, Skierso 199, Cunningsworth 1995, Hemsley 1997, Littlejohn 1998, Tomlinson 2003). The implication here is that the English language textbook employed in Saudi Arabia should embrace all of these components. In order to determine whether or not, from the perspective of the educators, the textbook used for teaching English in 6th grade public schools for boys covered these components, the questionnaire survey included a set of specific questions on this topic. These questions were not only designed to assess whether or not these components were included but whether or not they were presented in an effective way to facilitate the dual tasks of teaching and learning. The results are indicated in Table 8 below:
68 Table 8 Language components S
Strongly agree
agree
F
83
21
%
79.8%
20.2%
F
61
43
%
58.7%
41.3%
F
27
47
20
10
%
26.0%
45.2%
19.2%
9.6%
F
52
52
%
50.0%
50.0%
SECTORS
1 The book covers the main grammar items appropriate to sixth graders. 2 The book includes adequate materials for teaching vocabulary. 3 The book includes adequate material for pronunciation work. 4 The materials for teaching grammar, vocabulary, and pronunciation are graded in an appropriate manner.
disagree
Strongly disagree
Mean
Std. Deviatio n
3.79
0.40
3.58
0.49
2.87
0.91
3.50
0.50
Table 8 shows that there is a complete agreement on the statement that the book covers the main grammar items appropriate to sixth graders. Almost 80% of the participants strongly agree on the statement while the others agree. The mean score is 3.79 and the standard deviation is 0.40. Furthermore, the results show that the majority of the participants (58.7%) strongly agree and 41.3% agree that the materials for teaching vocabulary are adequate. The arithmetic mean reached 3.58 at a standard deviation of 0.49 (see table 8). Regarding the issue of whether the book includes adequate material for pronunciation work, the results show that there is a conflict between the participants about the adequacy of the materials for teaching pronunciation. Almost one third of the participants do not support the statement (19.2% disagree and 9.6% strongly disagree). However, the majority of the participants support the statement. Over 45% agree and 26%
69 strongly agree that the materials for teaching pronunciation are adequate. The mean is 2.87 and the standard deviation is 0.91 (see table 8). On whether or not the materials for teaching grammar, vocabulary, and pronunciation are graded in an appropriate manner, the results show there is a complete consensus that they are appropriately graded. Half of the participants strongly agree and the other half agree on the statement. The mean score reached 3.50 at a standard deviation of 0.50. As indicated above, opinions tended towards the positive. According to the majority of the participants, the textbook covered all of the components in question and quite effectively functioned as a teaching and learning aid. That does not mean to imply that the participants did not identify any area where there was room for improvement, as they did. According to approximately one-third of the participants, the materials for teaching pronunciation were not satisfactory and hence, did not effectively teach pronunciation. However, it is important to note that this was the only area over which critical opinions were voiced and, in general, the learning components were positively evaluated.
4.2.7 Socio-Cultural Context Appropriateness, discussed earlier, touched upon the socio-cultural context. It did not, however, explore the textbook in any depth from this particular evaluative perspective. Sheldon (1988) asserts the importance of checking whether or not a textbook "enshrine stereotyped, inaccurate, condescending or offensive images of gender, race, social class, or nationality (p.244). Accordingly, a set of questions were included which were designed to do just that. The results, as evidenced in Table 9 were mixed.
70 Table 9 Socio-Cultural Context S
SECTORS
Strongly agree
agree
disagree
Strongly disagree
1 The social and cultural contexts in the book are comprehensible.
F
55
49
%
52.9%
47.1%
2 The content of the book is free from stereotypical images and information.
F
92
12
%
88.5%
11.5%
3 Students can learn about the inner lives of the characters used in the book.
F
11
10
60
23
%
10.6%
9.6%
57.7%
22.1%
4 The book expresses positive views of ethnic origins, occupations, age groups, social groups and disability.
F
8
50
34
12
%
7.7%
48.1%
32.7%
11.5%
Mean
Std. Deviatio n
3.52
0.50
3.88
0.32
2.08
0.86
2.51
0.80
Table 9 shows that all the participants support the statement that the social and cultural contexts in the book are comprehensible; 52.9% strongly agree and 47.1% agree on the statement. No one disagree or strongly disagree. That can prove that the social and cultural components of the textbook in question are comprehensible. The arithmetic mean of this item reached 3.52 at a standard deviation of 0.50. Furthermore, table 10 also shows that a very large percentage of the participants (88.5%) strongly agree on the statement that the content of the book is free from stereotypical images and information. The rest of the participants agree on the statement and no one disagrees or strongly disagrees. That indicates that the book is completely free of any stereotypical images or information. The mean sore of this item was the highest in its category and reached 3.88 at a standard deviation of 0.32.
71 Regarding the issue of whether or not the book can allow students to learn about the inner lives of the characters used in the book, the results show that a high percentage of the participants do not agree that the student can learn about the inner lives of the characters used in the book. Only a little over 20% of the participant support the statement. The mean of this item is 2.08 at a standard deviation of 0.86 (see table 9). On whether the textbook in question expresses positive views of ethnic origins, occupations, age groups, social groups and disability or not, a large conflict among the participants can be observed. Over 48% of them agree on the statement and 7.7% strongly agree. On the other hand, 32.7% disagree and 11.5 strongly disagree. The arithmetic mean is 2.51 and the standard deviation is 0.80 (see table 9).
In general, the responses to the questions of this category have been varied and mixed. As regards the first question, whether or not the socio-cultural context was suitable and understandable, there was a general consensus that it was. This means that the textbook’s references were drawn from worldviews and socio-cultural environments with which the students were familiar. As for the second question, pertaining to whether or not the textbook promoted stereotypes, the participants unanimously maintained that it did not. The English language textbook used for 6th grade boys school the English language was free of any form or type of stereotypes, according to all participants. Freedom from stereotypes is one thing and the promotion of stereotypes is another. The first two questions may have established that the textbook in question is not founded upon and does not promote stereotypes but the last two questions established that it does not promote tolerance either. Approximately half of the participants opined that the textbook did not satisfy the objective of promoting positive views about others, whether those others represented different socio-economic, occupation, age, gender, national or ethnic groups. In other words, the last question affirmed the findings of the
72 third, solidifying the conclusion that the textbook in question does not promote tolerance. This was supported by Thein (2006) who found that the three English coursebooks taught at Myanmar Institute of technology included some negative information about some countries such as France, Spain, Italy and Holland. He claimed that such negative points of view can make people from these countries feel uncomfortable and upset.
4. 2.8 Skills Development Teaching language ultimately revolves around teaching students a specific set of skills, all of which are rooted in comprehension and communication. Teaching English to students means teaching them to comprehend what they read and understand what they listen to. It also means teaching students to express themselves and communicate their thoughts in spoken words and in writing. The linguistic proficiency which the Saudi public school systems aims to teach its students can only be attained if students grasp these particular skills. Therefore, to complement the earlier reviewed question sets which attempted the evaluation of the effectiveness of this particular book as a teaching and learning aid, six questions pertaining to the extent to which the textbook constructively satisfies the requirements of skills development were included in the survey questionnaire. The participants’ opinions are tabulated in Table 10.
73 Table 10 Skills Development S
SECTORS
Strongly agree
agree
disagree
Strongly disagree
1 The four skills are adequately covered.
F %
49 47.1%
53 51.0%
2 1.9%
2 There is material for integrated skills work
F %
19 18.3%
41 39.4%
40 38.5%
4 3.8%
3 Listening material is well recorded, as authentic as possible, and accompanied by background information, questions and activities.
F
7
27
58
12
%
6.7%
26.0%
55.8%
11.5%
F
7
89
8
%
6.7%
85.6%
7.7%
F
7
59
36
2
%
6.7%
56.7%
34.6%
1.9%
F
30
74
%
28.8%
71.2%
4 There is sufficient reading material. (There is a range of varied and interesting reading text that can engage students cognitively and effectively.) 5 There is sufficient material for spoken English ( e.g. dialogues, role-plays, etc.) 6 Writing activities are suitable in terms of length, degree of accuracy, and amount of guidance.
Mean
Std. Deviatio n
3.45
0.53
2.72
0.80
2.27
0.75
2.99
0.38
2.68
0.62
3.28
0.45
The results above show an extremely high percentage of the participants supporting that the four skills are adequately covered. Out of 104 participants, 49 participants strongly agree and 53 agree o. That means that over 98% of the participants favor the statement while 1.9 of the participants are against it. The arithmetic mean reached 3.45 at a standard deviation of 0.53 (see table 10). On whether there is material for integrated skills work or not, the results show that there is a conflict between the participants about that. Over 18% of the participants strongly agree and 39.4% agree with the statement. On the other hand, 38.5% of the participants disagree and 3.8% strongly disagree with the statement. The mean score is 2.72 and the standard deviation is 0.80.
74 Regarding the listening material of the book in question, the results show that the majority of the participants, over 67%, do not agree that the listening material is well recorded, as authentic as possible, and accompanied by background information, questions and activities. Only a little over 30% agree and strongly agree on the statement. The mean score is 2.27 and the standard deviation is 0.75. In table 10, over 85% of the participants agree and 6.7% strongly agree, while only 7.7% do not agree on the statement that there is sufficient reading material. This indicates a large consensus in favor of the reading material. The mean score is 2.99 at a standard deviation of 0.38. Concerning the issue of whether there is sufficient material for spoken English or not, table 11 shows that the majority of the participants support the statement. Over 56% agree and 6.7% strongly agree on the statement. On the other hand, over 36.5% do not support the statement. The results indicate there is again a conflict between the participants. However, most of the responses are in favor of the textbook. The mean score is 2.68 and the standard deviation is 0.62 (see table 10). Regarding writing activities, the results show that there is a complete consensus that activities are suitable in terms of length, degree of accuracy, and amount of guidance. Over two thirds of the participants agree on the statement and 28.8% strongly agree. No one disagrees with the statement. The mean reached 3.28 at a standard deviation of 0.45. Generally, the results, as evidenced in the above table are balanced. The textbook is not optimally designed to equally satisfy all the required skills but it does, nevertheless, generally satisfy the requirements of English language skills development. The four skills are, according to most of the participants, satisfactorily covered in the textbook. They are not, however, given equal weight. Focus tends to be significantly more on writing and reading, than on listening and conversation skills. Prior to criticizing this particular shortcoming, it is necessary to point out two facts. The first is that the textbook in
75 question is aimed towards 6th grade students in Saudi public schools, with the implication being that this group is still at the stage where they need to grasp a degree of mastery in vocabulary, grammar and writing. They need to grasp the language before they can engage in conversation. This does not mean that conversation is ignored as it is not. Instead, it is given less weight than are reading and writing skills.
4.2.9 Teachability A textbook is comprised of a set of lessons which teachers must teach and students must learn. The textbook in question is no exception. What is in question and what is a factor in the evaluation of the textbook, is the extent to which it helps teachers to minimize their preparation time. Teachers, despite their academic background and their professional experience, are bound by an official curriculum which they must satisfy in order to prepare students for successful testing and examination. Masuhara (1998) states "Teachers can … be said to be the central figures in the process of material development and yet their needs and wants are rarely given much consideration in coursebook development" (cited in Tomlinson, B. et al, 2001:88) Tomlinson B. et al (2001) added that: Although most of them (courses) consider the obvious need to help teachers to minimize their preparation time, very few provide help adapting the global course to specific situations, or cater for different teaching styles or personalities or make efforts to make the course interesting for the teachers (p.89).
The textbook in question represents the English language curriculum and the satisfaction of the requirements of this curriculum necessitates preparation on the part of the teachers. Actually, a poorly designed textbook does not facilitate lesson
76 preparation and indeed necessitates that teachers spend an inordinate amount of time on designing their lessons. A well-designed textbook, on the other hand, hardly requires that teachers spend time on preparing for their lessons. Given the importance of the role of the textbook in facilitating the process of teaching, it was necessary to evaluate the textbook in question from this particular perspective. Accordingly, three statements were included in the questionnaire survey. The results are given in Table 13, below: Table 11 Teachability S
SECTORS
Strongly agree
agree
disagree
Strongly disagree
1 The book helps teachers to minimize their preparation time.
F
53
49
2
%
51.0%
47.1%
1.9%
2 The book helps teachers exploit the activities to meet the students' expectations.
F
42
50
12
%
40.4%
48.1%
11.5%
3 The book helps teachers cater for mixed ability students and classes of different sizes.
F
23
45
30
6
%
22.1%
43.3%
28.8%
5.8%
Mean
Std. Deviati on
3.49
0.53
3.28
0.66
2.81
0.84
Table 11 shows that a high percentage of the participants support the statement that the book helps teachers to minimize their preparation time, 51% of the participants strongly agree and 47.1% agree. Only 1.9% of the participants disagree on the statement. The arithmetic mean is 3.49 at a standard deviation of 0.53. 4.2.9.2 The book helps teachers exploit the activities to meet the students' expectations. On whether or not the book helps teachers exploit the activities to meet the students' expectations, table 12 shows again a large agreement. Over 88% of the participants perceive that the book helps teachers exploit the activities to meet the
77 students' expectations. Only 11.5% of the participants disagree with the statement. The mean score is 3.28 and the standard deviation is 0.66. Regarding the issue of whether the book helps teachers cater for mixed ability students and classes of different sizes, the results show that most of the participants support the statement, 43.3% strongly agree and 22.1% agree. On the other hand, only about one third of the participants do not support the statement, 28.8% disagree and 5.8 strongly disagree. The arithmetic mean is 2.81 at a standard deviation at 0.84 (see table 11). In general, the greater majority of the participants opined that the textbook in question greatly facilitated the ability of teachers to prepare their lessons and, indeed, to satisfy their objective of effective teaching. The implication here is that the textbook functions as a very good guide for teaching English to sixth grade boys in Saudi Arabian schools. This does not mean, however, that there is no room for improvement since approximately one-third of the participants believed that the textbook did not aid teachers in catering for mixed ability students and classes of different sizes. The fact is that students cannot be defined as a homogenous group and given that the textbook is designed for use in all sixth grade classes across the Kingdom’s boys' schools, classrooms cannot be identified as homogenous either. This may necessitate that teachers adopt varied teaching strategies in order to cater to different class sizes and needs.
4.2.10 Flexibility It worth recall that Sheldon (1988), Tomlinson B. et al (200) and many scholars in the field assert the importance of evaluating the flexibility of a textbook. Sheldon (1988), for example, asserts the importance of evaluating whether a textbook can be "exploited or modified as required by local circumstances or it is too rigid in format, structure and approach" (p. 245). Accordingly, a category of four items were included in
78 the survey questionnaire to test the flexibility of the textbook in question and the results obtained are indicated below. Table 12 Flexibility S
SECTORS
Strongly agree
agree
disagree
Strongly disagree
1 The book is appealing and useful to the students.
F
34
54
16
%
32.7%
51.9%
15.4%
2 The book caters for different levels of formality.
F
2
19
73
10
%
1.9%
18.3%
70.2%
9.6%
3 The book provides opportunity for teachers and students to localize activities.
F
15
80
9
%
14.4%
76.9%
8.7%
4 The book caters for different preferred learning styles.
F
19
61
14
%
27.9%
58.7%
13.5%
Mean
Std. Deviatio n
3.17
0.67
2.12
0.58
3.05
0.47
2.14
0.62
Table 12 shows that the majority of the participants support the statement about whether or not the book is appealing and useful to the students. One third of the participants strongly agree and almost 52% agree that the book is appealing and useful to students. On the other hand, only 15.4% of the participants do not agree on the statement. The arithmetic mean is 3.17 and the standard deviation is 0.67. Regarding the issue of whether or not the book caters for different levels of formality, the results indicate that a high percentage of the participants do not support the statement. Over 70% disagree and almost 10% strongly disagree that the book can cater for different levels of formality. Only a little more than 20% support the statement. The mean is 2.12 and the standard deviation is 0.58 (see table 12). Concerning the issue of whether the book provides opportunity for teachers and students to localize activities, the results show that a large percentage of the
79 participants, over 90%, support the statement. Only 8.7% of the participants do not agree with the statement. The mean score is 3.05 at a standard deviation of 0.47. Furthermore, the results show that the majority of the participants do not support the statement that the book caters for different preferred learning styles. Over 58% of the participants disagree and 13.5% strongly disagree with the statement that the book can cater for different learning styles. On the other hand, only 27.9% of the participants agree on the statement. The arithmetic mean is 2.14 at a standard deviation of 0.62 (see table 12).
As illustrated in the above, four questions were asked and the results tended to be mixed. Regarding the issues of whether or not the textbook was useful and appealing to students and whether or not it helps localize the activities of both students and teachers, the results were positive and definitely indicated that the textbook contributed significantly to the satisfaction of these two functions. As regards whether or not the linguistic style was varied, so as to appeal to different levels of formality and whether or not it facilitates the exploitation of different learning styles, the results were negative. The textbook imposes learning and teaching style upon students and teachers, leaving them little option for the use of the one which they prefer. Added to that, the textbook tends towards the formal style and contains, little, if any, informality. The implication here is that insofar as variation of appeals (including style, learning strategy, etc) are concerned, the textbook leaves much room for improvement. The results were supported by Tomlinson B. et al (2001) who found that most of current courses "give little opportunity or encouragement to adapt the materials to the needs, wants, personalities, or styles of the learners or teachers" (p.83).
80 4.2.11 Teaching Methods Integral to the evaluation of the textbook in question is the determination of the extent to which the teaching methods upon which it is founded are studentcentered. In fact, there is a discernible movement away from teacher-centered methodologies to student-centered ones, with the rationale being that the latter has established itself as a more effective approach to teaching and learning. The fact is, however, that not all teachers and educators recognize this and, accordingly, adhere to the teacher-centered methodology of teaching. In acknowledgement of the aforementioned, the questionnaire sought the evaluation of the textbook under study from the perspective of whether or not it embraced contemporary student-centered models of learning and, indeed, whether it gave teachers the option of utilizing a student-centered model by allowing students to talk more than teachers and allowing various class activities. The research findings as pertaining to this particular aspect of investigation are contained in Table 17 below: Table 13 Teaching Strategy S
Strongly agree
SECTORS
1 The teaching methods used in the book are the latest in the field.
agree
disagree
Strongly disagree
F
32
67
5
%
30.8%
64.4%
4.8%
2 The methods used are student-centered.
F
5
22
71
6
%
4.8%
21.2%
68.3%
5.8%
3 The methods used allow students to talk more than teachers.
F
12
80
12
%
11.5%
76.9%
11.5%
4 The methods used allow various class activities.
F
5
16
50
33
%
4.8%
15.4%
48.1%
31.7%
Mean
Std. Deviatio n
2.25
0.53
2.25
0.45
2.00
0.48
1.93
0.81
81 With regards to the teaching methods used in the book, the results show that an extremely high percentage of the participants do not support the statement that the teaching methods used in the book are the latest in the field. Over 64% of the participants disagree on the statement. Furthermore, about 5% strongly disagree on the statement. On the other hand, only 30.8% agree that the teaching methods used in the book are the latest in the field. The mean score is 2.25 and the standard deviation is 0.53 (see table 13). Furthermore, the results show that the majority of the participants do not support this statement that the methods used are student-centered. Out of 104 participants, 71 participants disagree and 6 strongly disagree. On the other hand, only 12 participants agree on the statement. The arithmetic mean is 2.25 and the standard deviation is 0.45. In addition, the results show that an extreme percentage of the participants do not support the statement that the methods used allow students to talk more than teachers. Almost 77% of them disagree and 11.5% strongly disagree. On the other hand, only 11.5% agree on the statement. The mean is 2.00 and the standard deviation is 0.48 (see table 13). On whether or not the methods used allow various class activities, the results indicate that the majority of participants do not support this statement as well. Over 48% disagree on the statement and almost 32% strongly disagree. On the other hand, only one fifth of the participants support the statement. The mean of this statement is 1.93 and the standard deviation is 0.81(see table 13).
As indicated above, the results tended towards the negative, strongly suggesting the imperatives of revision. Not only did the majority believe that the textbook unfolded in accordance with outmoded teaching and learning strategies but an overwhelming majority of participants maintained that the book is teacher centered, does not allow for student participation and quite effectively constraints any, and all,
82 opportunities for class activities which would promote the teaching and learning of English. Indeed, the results of the questionnaire established the fact that this textbook adhered to a teaching and learning model which centralized rote memorization, as opposed to learning through activities. It is important to highlight the fact that this was the opinion voiced by the greater majority of the participants. These results were supported by Tomlinson B. et al (2001) who state that: The most obvious pedagogic feature of many of courses is that they are teaching centered … and … seem to be a reaction against the freer, open ended, learner centered days of the communicative approach (p. 87).
4.2.12 Practice and Testing Needless to say, one of the more important functions of any textbook is both to teach and to effectively prepare students for success on examinations. It is, thus, that the majority of textbooks contain exercises and preparation tests following each lesson covered. Even though the inclusion of exercises is an important determinant of the textbook’s usefulness and practicality as a teaching tool, the presentation and nature of these exercises are even more important determinants. Not only must the exercises be representative of the lessons but they must be clearly presented and, ideally formatted, presented and explained as they would be in an exam. In consideration of the aforementioned, therefore, the questionnaire included a set of seven questions whose purpose was to examine the degree to which the 6th grade English language textbook satisfies the objectives of preparation and testing. The results of this particular question set are illustrated in table 15 below.
83 Table 14 Preparedness and Testing S
SECTORS
Strongly agree
agree
disagree
Strongly disagree
1 The book provides a variety of meaningful and mechanical exercises and activities to practice language items and skills.
F
31
71
2
%
29.8%
68.3%
1.9%
2 It provides communicative exercises and activities that help students carry out their communicative tasks in real life.
F
43
41
20
%
41.3%
39.4%
19.2%
3 Every exercise has a clear direction.
F %
46 44.2%
58 55.8%
4 There is a reasonable and appropriate number of exercises.
F
59
45
%
56.7%
43.3%
5 The tests are valid and contain correct language.
F
59
45
%
56.7%
43.3%
6 The book provides periodical revisions for diagnostic purposes.
F
25
79
%
24.0%
76.0%
7 The book provides models for final achievement tests.
F %
22 21.2%
74 71.2%
8 7.7%
Mean
Std. Deviatio n
3.27
0.49
2.22
0.77
3.44
0.49
3.56
0.49
3.56
0.49
3.24
0.42
3.13
0.52
The table above shows that an extreme percentage of the participants support the statement that the book provides a variety of meaningful and mechanical exercises and activities to practice language items and skills. Almost 30% of them strongly agree and 68.3% agree on the statement. On the other hand, only 2 participants, 1.9%, do not agree
84 on the statement. The arithmetic mean is 3.27 and the standard deviation is 0.49 (see table 14). On whether the book provides communicative exercises and activities that help students carry out their communicative tasks in real life or not, the results indicate that over 60% of the participants do not support the statement. Out of 104 participants, 20 participants strongly disagree on the statement and 39 participants disagree. On the other hand, only 41 participants agree on the statement. Therefore, it can be said that, for many participants, although the book provides a variety of exercises and activities, it does not provide enough communicative ones. The mean reaches 2.31 at a standard deviation of 0.77. Concerning the issue of whether or not every exercise in the textbook in question has a clear direction, the results indicate a complete consensus on the statement. Over 44% of the participants strongly agree and almost 56% agree on the statement that every exercise has a clear direction. The arithmetic mean is 3.44 and the standard deviation is 0.49 (see table 14). On whether or not there is a reasonable and appropriate number of exercises provided in the book, the results indicate that almost 57% strongly agree and over 43% agree on the statement. The mean is 3.56 and the standard deviation is 0.49.
Again, the table shows that all participants support the statement that tests are valid and contain correct language. Out of 104 participants, 59 participants strongly agree and 45 agree. No negative responses are detected. Therefore, it can be said that the tests provided in the textbook are valid and contain correct language. The mean score reaches 3.56 and the standard deviation is 0.49 (see table 14). Regarding the issue of whether the book provides periodical revisions for diagnostic purposes, table 15 shows that all the participants support the statement as well,
85 24% of them strongly agree and 76% agree. The mean is 3.24 and the standard deviation is 0.42 (see table 14). Finally, the results indicate that there is an extreme percentage of the participants support the statement that the book provides models for final achievement tests. Over 92% of the participants support the statement while only 7.7% disagree on it. The arithmetic mean is 3.13 and the standard deviation is 0.52.
As noted above, the participants tend towards a largely positive view of the textbook as a tool for practice and testing. The greater majority of participants have positive views as regards the attention the textbook pays to practice and testing, maintaining that the exercises contained therein are varied, clearly explained and representative of the preceding lessons. In addition, the test included functions as a model for the final exams which students are required to sit for at the end of the school year. Within the context of the stated, it is possible to affirm that the textbook fulfills the criteria established for practice and testing except for the inclusion of communicative exercises and activities that help students carry out their communicative tasks in real life. This is very unfortunate since communicative exercises such as role play, games, puzzles and group and pair work discussions can help students improve their communication skills. Hedge (2000) states that "the ability to communicate effectively in English is now a well-established goal in ELT" (p. 44).
4.1.3 Results of Per-Category Analysis In this part, the results answer the questions asked in this research about the quality of the appearance of the book, design and illustration, accompanying materials, objectives, topic appropriateness, language components, socio-cultural context, skills development, teachability, flexibility, teaching method, and practice and testing.
86 The total means and standard deviations were calculated from the means and standard deviations of the sub-items, which were the explanation of the main parts. Table 16 shows the mean and standard deviation for each of the main parts of the questionnaire.
Table 15 Arithmetic means and standard deviations of the twelve categories of evaluative criteria Main categories
N
Mean
Std. Deviation
A- General Appearance
104
2.9533
0.2935
B- Design and Illustration
104
2.7692
0.4970
C- Accompanying Materials
104
3.2981
0.5037
D- Objectives
104
3.2615
0.3507
E- Topic Appropriateness
104
2.9351
0.4151
F- Learning Component
104
3.4399
0.3856
G- Socio-cultural Contexts
104
3.0048
0.3987
H- Skills Development
104
2.9022
0.3728
I- Teachability
104
3.1987
0.5347
J- Flexibility
104
2.6250
0.3620
K- Teaching Methods
104
2.3389
1.2473
L- Practice and Testing
104
3.2212
0.3330
Table 15 shows that the general tendency of the participants' perceptions was toward supporting the Sixth Grade English Language Textbook for Saudi boys' School. Out of the 12 categories of the evaluative questionnaire, 6 categories scored arithmetic means of 3 or more. These categories are accompanying materials, objectives, language components, socio-cultural contexts, teachability and practice and testing.
87 The table also shows that there is a clear dispute over the general appearance, design and illustration, topic appropriateness, skills development, and flexibility. These are the five categories that had arithmetic means between 2.51 and 2.99. These are considered to be satisfied but still need some improvement. The table also shows that only one category that is not satisfied in the textbook, which is teaching methods. This had an arithmetic mean less than the average 2.50. The table also shows that the group of statements on language components had the greatest mean, while the group on teaching methods had the lowest mean. The table above also shows that the statements of the group on general appearance were the ones answered most consistently, but with a very wide range of answers for the group on teaching methods (see table 15).
4.2.3 Equality of the Means of the Two Populations (Teachers and Supervisors) In order to test the equality of the means of teachers and supervisor with respect to the 12 main categories of the study, an independent samples t-test was conducted. The results are tabulated below.
88 Table 16 Equality and Variance of the means of teachers and supervisors Category F Sig. job N
Mean
Std. Deviation
A- General Appearance
.124
.725
teacher Supervisor
93 11
2.9524 2.9610
.2906 .3325
B- Design and Illustration
1.556
.215
teacher Supervisor
93 11
2.7404 3.0130
.4761 .6209
C- Accompanying Materials
.303
.583
teacher Supervisor
93 11
3.2876 3.3864
.5040 .5168
D- Objectives
.283
.596
teacher Supervisor
93 11
3.2720 3.1727
.3503 .3580
E- Topic Contents
1.135
.289
teacher Supervisor
93 11
2.9194 3.0682
.4173 .3888
F- Language Contents
.115
.736
G- Social and Cultural Contexts
8.738
.004
teacher Supervisor teacher Supervisor
93 11 93 11
3.4516 3.3409 2.9812 3.2045
.3817 .4224 .3672 .5897
3.859
.052
teacher Supervisor
93 11
2.8817 3.0758
.3540 .4908
I- Teachability
.058
.810
teacher Supervisor
93 11
3.1720 3.4242
.5353 .4965
J- Flexibility
.000
.999
teacher Supervisor
93 11
2.5806 3.0000
.3341 .3873
K- Teaching Methods
.285
.595
teacher Supervisor
93 11
2.3172 2.5227
1.3022 .6170
L- Practice and Testing
.004
.949
teacher Supervisor
93 11
3.2320 3.1299
.3283 .3750
H- Language Skills
Note: Difference is significant at p < .05
89 Table 17 shows the attitudes of supervisors and teachers towards the Sixth Grade English Language Textbook for Saudi Boys' Schools with respect to the 12 main categories. The results show that there are no differences in the attitudes of teachers and supervisors towards the textbook in question except on the flexibility of the textbook. The T-value reached 3.872. This has a statistical indication on level less than 0.05. The table shows that the arithmetic mean of teachers reached (2.5806) at standard deviation (0.33), whereas the arithmetic mean of supervisors reached (3.00) at standard deviation (0.38). So, the attitudes of teachers according to the arithmetic mean are less than supervisors’ attitudes. This difference can be attributed to the different nature of jobs of teachers and supervisors.
4. 5 Conclusion In summary, this chapter provided insights to the data analysis and results of the study. The following is a summary of the research findings. •
On the Per-Statement Analysis
As noted above, the first set of computations was those of the frequencies, percentages, means and standard deviations for each and every statement in the questionnaire. This part aims at defining the points of strength and weakness in the textbook.
The results show that, out of the 63 items in the questionnaire, 48 items elicited a very positive response. This means that positive responses were given for over 79% of the items in the survey. This speaks well of the textbook, because teachers and supervisors are a very good index for evaluating the said textbook as they are the ones who are in direct contact with the process of teaching and learning.
90 On the other hand, the 13 statements with lowest means, less than 2.50, are posted just below. This means that these statements mention those desired qualities of the book that may not be present in the current textbook from the point of view of the participants. These are considered as the weak points of the textbook in question.
Table 17 Statements with the lowest means No. Statement 1. The book has an appropriate glossary.
Mean 1.60
2.
The book has a complete bibliography.
1.18
3.
The illustrations stimulate students to be creative.
1.81
4.
The topics allow students to think critically.
1.65
5.
Students can learn about the inner lives of the characters used in the book.
2.08
6.
Listening material is well recorded, as authentic as possible, and accompanied by background information, questions and activities.
2.27
7.
The book caters for different levels of formality.
2.12
8.
The book caters for different preferred learning styles.
2.14
9.
The teaching methods used in the book are the latest in the field.
2.25
10.
The methods used are student-centered.
2.16
11.
The methods used allow students to talk more than teachers.
2.00
12.
The methods used allow various class activities.
1.93
13.
The book provides communicative exercises and activities that help students carry out their communicative tasks in real life.
2.22
In addition to the items mentioned above, two other items were not supported by nearly half the participants. The first is that whether or not the illustrations are varied and attractive. The second is that whether or not the textbook expresses positive views of
91 ethnic origins, occupations, age groups, social groups and disability. So, these should be taken into consideration as a big percentage of the participants did not support them.
With regards to standard deviation, it is clear that since there were only 4 choices in the answers, it is quite understandable that the standard deviations did not scatter too much. However, after looking at the list of statements, the following are the statement with the highest standard deviations.
Table 18 Statement with the highest standard deviations No.
Statement
Std. Deviation
1.
The cover is informative and attractive.
2.
The illustrations are varied and attractive.
3.
The illustrations stimulate students to be creative.
4.
The topics encourage students to express their own views.
5.
The book includes adequate material for pronunciation work.
6.
Students can learn about the inner lives of the characters used in the book.
0.86
The book expresses positive views of ethnic origins, occupations, age groups, social groups and disability.
0.80
7
8
There is material for integrated skills work.
9
Listening material is well recorded, as authentic as possible, and accompanied by background information, questions and activities.
10
The book helps teachers cater for mixed ability students and classes of different sizes.
11
The methods used allow various class activities.
12
It provides communicative exercises and activities that help students carry out their communicative tasks in real life.
0.70 0.80 0.97 1.00 0.91
0.80
0.75
0.84 0.81 0.77
92 The above table shows the statements with the most variable or “spread-out” answers. Again, the standard deviations are one or less because of the nature of the scoring of the survey.
On the other hand, table 20 shows the statements with the least values of standard deviation.
Table 19 Statements with the least standard deviations. No.
Statement
1.
The book avoids potentially embarrassing or disturbing topics.
2.
The content of the book is free from stereotypical images and information.
3.
There is sufficient reading material.
Std. Deviation 0.32 0.32 0.38
The tables above means that the most consistent sets of replies were to the statements on whether or not the book avoids potentially embarrassing or disturbing topics and on whether or not the content of the book is free from stereotypical images and information. The answers to this hardly varied, as we can clearly see.
• On The Per-Category Analysis The results revealed that all the categories of characteristics defined earlier in this research are satisfied in the Sixth Grade English Language Textbook for Saudi Boys' Schools except the category on teaching methods, since it was the only category that had an arithmetic mean less than the average (2.50).
The results also revealed that although the criteria of the general appearance, design and illustration, topic appropriateness, skills development, and flexibility are satisfied in the book but still they need to be improved.
93 • On the Equality and Variance of Means of Teachers and Supervisors The independent t-test conducted to compare the responses of teachers and the responses of supervisors show that there is no significant difference between the responses of the two groups of participants with respect to the main categories of the questionnaire except on the category about the flexibility of the textbook. This can be attributed to the difference in the nature of the jobs they do.
94
CHAPTER FIVE Summary, Conclusion, Recommendation and Suggestions
5.1 Introduction This chapter consists of four sections: summary, conclusion, recommendations and suggestions. First, the summary presents an overall account for all parts of the research. Second, the conclusion presents the study's findings in light of the researcher hypotheses.. Third, it provides recommendations for modifying the textbook at hand and finally, it ends with suggestions for future research 5.2 Summary The purpose of this study was to evaluate the quality of the Sixth Grade English Language Textbook for Saudi Boys' Schools. A survey questionnaire was used in this study to elicit the perspectives of 93 English language teachers and 11 supervisors in Riyadh Educational Zone about the textbook in question. The questionnaire consisted of 63 items grouped under 12 main categories: the general appearance, design and illustration, accompanying materials, objectives, topic appropriateness, language components, socio-cultural contexts, skills development, teachability, flexibility, teaching methods, and practice and testing. The data were subjected to analysis through descriptive statistics. Three different types of computations were done: a per-item analysis, a per-category analysis, and an independent samples t-test to clarify the equality and variance between the two groups of the population (teachers and supervisors). The findings of this research study revealed important points related to the characteristics of a good textbook. The findings were generally in favour of the textbook except for the teaching methods and some other sub-items. Out of 63 items in the
95 questionnaire, only 13 items had arithmetic means less than 2.50 (the average). The category that had the highest mean was the one on language components, while the category that had the lowest mean was the one on teaching methods. The findings also revealed that there were no significant differences between the means of the two populations of the study (teachers and supervisors) except on the flexibility of the textbook and this can be attributed to their different jobs.
5.3 Conclusion Based on the results obtained from the per-statement analysis, per-category analysis and the independent samples t-test, the following conclusions are drawn: 1. The Sixth Grade English Language Textbook for Saudi Boys' Schools satisfies teachers’ and supervisors’ expectations regarding its general appearance. However, an appropriate glossary should be added. 2. The textbook in question satisfies teachers’ and supervisors’ expectations regarding its design and illustrations. However, it lacks illustrations that are varied and attractive and that can stimulate students' creativity. 3. The textbook in question satisfies teachers’ and supervisors’ expectations regarding its accompanying materials. 4. The textbook in question satisfies teachers’ and supervisors’ expectations regarding its objectives. 5. The textbook in question satisfies teachers’ and supervisors’ expectations regarding its topic appropriateness. However, there is a lack for topics that can stimulate students' critical thinking. 6.
The textbook in question satisfies teachers’ and supervisors’ expectations regarding its language components. According to an extremely high
96 percentage of the participants, all the language components are presented and graded in a satisfactory manner. 7. The textbook in question satisfies teachers’ and supervisors’ expectations regarding its socio-cultural contexts. However, it lacks information about the inner lives of the characters used. In addition, the textbook failed to express positive views of ethnic origins, occupations, age groups, social groups and disability according to almost half of the participants. 8. The textbook in question satisfies teachers’ and supervisors’ expectations regarding skills development. However, the listening material does not satisfy the participants' expectations. 9. The textbook in question satisfies teachers’ and supervisors’ expectations regarding its teachability. 10. The textbook in question satisfies teachers’ and supervisors’ expectations regarding its flexibility. However, it does not cater for different levels of formality and different learning styles. 11. The textbook in question does not satisfy teachers’ and supervisors’ expectations regarding its teaching methods. According to the majority of the participants, the teaching methods are not up-to-date, not student-centered, do not allow students to talk more than teachers, and does not allow various classroom activities. 12. The textbook in question satisfies teachers’ and supervisors’ expectations regarding the practice and testing. However, it failed to provide communicative exercises and activities that can help students carry out their communicative tasks in real life.
97 5.4 Recommendations Based on the results obtained from this study, the following points are recommended: 1. Adding an appropriate glossary at the end of the textbook for the benefit of both teachers and students. 2. Including a variety of attractive illustrations that can stimulate creativity. 3. Adding topics that can provoke students' critical thinking, encourage them to generate their ideas and leave room for making judgments on others' opinions. 4. Including information about the inner lives of the characters used in the book as this can allow students to discuss the characters' personalities, behaviours, problems, attitudes and ways of thinking. In addition, these characters can be examples for the students in their real lives. 5. Inserting context that can express positive views of ethnic origins, occupations, age groups, social groups and disability. 6. Replacing the listening material with a better one that is well recorded, more authentic, and accompanied with background information, questions, and activities. 7. Allowing some flexibility by providing opportunity to practice language formally and informally so that students can practice various degrees of language such as simple greetings, short telephone exchanges, polite requests and formal invitations. 8. Students vary in their preferred learning styles. The textbook emphasises analytical learning that depends mainly on practice and memorization and does not take account of the fact that some students may prefer experiential or kinaesthetic learning, for example.
98 9. It was clear from the research results that the textbook fails to satisfy the teachers' and supervisors' expectations regarding its teaching methods. So, it is necessary to update these methods and make use of freer, student-centered approaches that can engage students affectively through excitement, fun and emotion. Communicative approaches that can allow various activities are highly recommended. Only then, student can be encouraged to talk and actually use the language. The attention should be on the performance and meaning rather than competence and accuracy. 10. Adding communicative exercises and activities such as group and pair work, games, puzzles and role play as these can help students carry out their communicative tasks in real life.
5.5 Suggestions for Further Studies 1. A content analysis study for the same textbook evaluated in this study is needed as it may uncover more valuable results. 2. An extension of this study to evaluate the English language textbooks recently introduce to be taught at the Saudi intermediate schools are required. 3.
Empirical evaluative studies to evaluate the English textbooks in Saudi girls' schools are also suggested.
4. A further study might be conducted to investigate the whole program for teaching English as foreign language in Saudi Arabia.
99
References Ali, F.M. (1983). Evaluating the English Language Textbook studied at the Second Grade Intermediate Schools in Saudi Arabia. Unpublished doctoral dissertation, Umm Al-Qura University. Makkah, Saudi Arabia.AlHijailan, T. (2003) Teaching English in Saudi Arabia. Riyadh: Aldar Alsawlatiah. ____________ (1999) Evaluation of English as a Foreign Language Textbook for Third Grade Secondary Boys’ Schools in Saudi Arabia. Unpublished doctoral dissertation, Mississippi State University, Mississippi, USA. Al-Mofarreh, et al. (2005) English for Saudi Arabia: Sixth Grade Elementary. Riyadh: Ministry of Education. Aljaser, Afaf M. (1989). An Analysis of the English Language Textbooks Taught at the FirstYear Boy and Girls' Secondary Schools: A Comparative Study .Unpublished MA thesis, King Saud University. Riyadh, Saudi Arabia. Al-Sadan, I. A. (2000) Educational assessment in Saudi Arabian schools. Unpublished PhD thesis, Cardiff University. Al-Subahi, A. A. O. (1989) A Communicative-Functional English Curriculum For Intermediate School In Saudi Arabia: a model for English syllabus design and implementation. Unpublished PhD thesis, Department of Education, University of Edinburgh. Atkins, A. (2001). An evaluation of the coursebook used for oral communication one at privately funded senior high school in Japan. Retrieved on March 11, 2006, from http:/www.cels.bham.ac.uk/resources/essays/atkins3.pdf Brown, J.D. and Rogers, T.S. (2002) Doing second language research. Oxford: Oxford University Press.
100 Bruder, M. (1978). Evaluation of Foreign Language Textbooks. In H. Madsen & J. Brown (Eds.),
Adaptation in language teaching, 209-218. Massachusetts:
Newbury House Publishers, Inc. Byrnes, H. (2000). Shaping the discourse of a practice: The role of linguistics and psychology in language and teaching. The Modern Language Journal, 84(4). Carter, R and Nunan, D. (2001) The Cambridge Guide to Teaching English to Speakers of Other Languages. Cambridge: Cambridge University Press. Chambers, F. (1997). Seeking consensus in textbook evaluation. ELT Journal, 51(1), 2935. Crandall, E. and Basturkmen, H. (2004). Evaluating pragmatics-focused materials. ELT Journal, 58(1), Cunningsworth, A. (1984). Evaluating and selecting EFL teaching materials. London: Heinemann. Cunningsworth, A. (1995). Choosing your Coursebook. London: Macmillan: Heinemann. Davies, A. (2006). What do learners really want from their EFL courses?. ELT Journal, 60(1), 3-12 Duff, P.A., Wong, P. and Early, M. (2002) Language learning for work and life: The linguistic socialization of immigrant Canadians seeking careers in healthcare. The Modern Language Journal, 86(3). Ellis, R. (1997). The empirical evaluation of language teaching materials. English Language Teaching Journal, 51(1), 36-42 Eriksoussy, Zubaidah M. (1993). Evaluating the English Language Textbook Studied at the First Year Intermediate Schools in Saudi Arabia. Unpublished MA thesis, King Saud University. Riyadh, Saudi Arabia.
101 Everitt, B. S. (2002). The Cambridge Dictionary of Statistics. Cambridge, UK: Cambridge University Press. Garinger, D. Textbook Evaluation. Retrieved July 14, 2007, from http://www.teflwebj.org/v1n1/garinger.html Graves, K. (2000). Designing Language Courses. Canada: Newbury House Harmer, J. (2001) The Practice of English Language Teaching. Essex: Pearson Education Limited. Hayes, N. and Schrier, L. (2000) Encouraging second language literacy in the early grades. Hispania, 83(2). Hedges, T. (2000). Teaching and Learning in the Second Language Classroom. Oxford: Oxford University Press. Hemsley, M. (1997). The evaluation of teachers' guides design and application. ELT Journal, 3 (1), 72-83. Hutchinson, T. and Water, A. (1987) English for Specific Purposes: A learning centred approach. Cambridge: Cambridge University Press. Johnson, K. (2006). A step forward: Investigating expertise in materials evaluation. ELT Journal, 60 (3) 1-7. Khafaji, A. (2004). An evaluation of the materials used for teaching English to the second secondary level in male public high school in Saudi Arabia. Unpublished MA thesis, University of Exeter. Exeter, UK. Key, J. P. (1997). Reliability and Validity. Retrieved March 14, 2006, from http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage18. htm Littlejohn, A. (1998) The analysis of language teaching materials: inside the Trojan Horse. In Tomlinson, B. (ed.) (1998) Materials Development in Language Teaching. Cambridge: Cambridge University Press.
102 Lynch, B. K. (1996) Language Program Evaluation: Theory and Practice. Cambridge: Cambridge University Press. McGrath, I. (2002). Materials Evaluation and Design for Language Teaching. Edinburgh: Edinburgh University Press. Ministry of Education (1970) Educational Policy of Saudi Arabia. Riyadh, Saudi Arabia: Ministry of Education. Ministry of Education, General Directorate of Curricula (2002) English for Saudi Arabia. Jeddah, Saudi Arabia: Almadina Almunawara Press. Nitta, R & Gardner, S. (2005). Consciousness-raising and practice in ELT coursebooks. ELT Journal, 59(1), 3-13. Nunan, D. (1988). Syllabus Design. Oxford: O.U.P. Nunan, D. (1988) Learner-Centered Curriculum. Cambridge: Cambridge University
Press. Olivares-Cuhat, G. (2001). An evaluative approach for L2/FL Composition Textbooks. Academic Exchange Quarterly, 5(3), pp. 74-77. Palmer, H. (1922). The Principles of Language Study. London: O.U.P. Reinders, H & Lewis M. (2006) An evaluative checklist for self-access materials. ELT Journal, 60 (3) 272-278. Sheldon, L. (1988). Evaluating ELT textbooks and materials. English Language Teaching Journal, 42(4), 237-246. Skierso, A. (1991). Textbook Selection and Evaluation. In M. Celce-Murcia (2nd ed.), Teaching English as a second or foreign language, 432-453. Spielmann, G. and Radnofsky, M.L. (2001) Learning language under tension: New directions from a qualitative study. The Modern Language Journal, 85(2).
103 Thein, N. (2006) Evaluating the suitability and effectiveness of three English coursebooks at Myanmar Institute of Technology. Unpublished MA thesis, University of Thailand. Thailand. Tomlinson, B. (1998). Comments on Part C. In Tomlinson, B. (ed) (1998) Materials Development in Language Teaching. Cambridge: Cambridge University Press. Tomlinson, B. (1999) Developing Criteria for Materials Evaluation. In IATEFL, No. 147, 10-13. Tomlinson, B. et al. (2001). ELT Courses for Adults. ELT Journal, 55(1) 80-101. Tomlinson, B. (2003) Materials Evaluation. In Tomlinson, B. (ed) (2003) Developing Materials for Language Teaching. London and New York: Continuum. Tu'eimah, R. A. (1985). A guide for preparing Arabic language teaching programs. Arabic Language Institute. Umm Al-Qura University, Makkah, Saudi Arabia. Ur, Penny, (1996). A Course in Language Teaching: Practice and Theory. Cambridge: Cambridge University Press Weir, C. J. and Roberts, J. (1994) Evaluation in ELT. Oxford: Blackwell. Williams, D. (1983). Developing criteria for textbook evaluation. ELT Journal, 37(2), 251-255. Witte, R. S. (1985). Statistics (2nd ed.). New York: Holt, Rinehart and Winston. Witte, R. & Witte, J. (2004). Statistics (8th ed.). New York: Harcourt Brace & Co.
104
Appendix A (Continued)
Dear teacher/supervisor, Assalamu Alaikum. I am conducting a research study entitled “An Evaluation of the Sixth Grade English Textbook for Saudi Boys' Schools”. This questionnaire is the main research instrument. Please read every statement carefully and circle the choice that you find most suitable. I really appreciate your cooperation and participation.
The researcher/ Abdulrahman Alamri
105 An Evaluation of the Sixth Grade English Textbook for Saudi Boys' Schools. Respondent Number: ____ PERSONAL INFORMATION
Teacher’s Name (optional):_____________________________________ School: (Optional) ___________________________________ Qualification:
Diploma
B.A
Nationality:
Saudi
Non-Saudi
B.Ed
M.A
M.Ed
Experience in teaching English: First year
2-5 years
More than 5 years
INSTRUCTION To respond to this questionnaire, please check ( 1 (strongly disagree) 2 (disagree)
√ ) in the appropriate choice as follows: 3 (agree)
4 (strongly agree)
A- General Appearance No. 1 2 3 4 5 6 7
Statement The cover is informative and attractive. The font size and type used in the book are appropriate for sixth graders. There is an informative orientation page. The book has a complete and detailed table of contents. Every lesson is given an appropriate title. The book has an appropriate glossary. The book has a complete bibliography. B- Design and Illustration
1 2 3 4 5 6 7
There is a variety of design to achieve impact. There is enough white space to achieve clarity. There is consistency in the use of headings, icons, labels, italics, etc. The illustrations are varied and attractive. The illustrations stimulate students to be creative. The illustrations are functional. The illustrations facilitate students' visualizations without imposing complete visual images.
1 2 3 4
106
C- Accompanying Materials 1 2 3 4
Cassettes that accompany the book are suitable. The CD-ROM that accompanies the book is suitable. The posters and flash cards that accompany the book are suitable. The teacher's book that accompanies the book is informative. D- Objectives
1 2 3 4 5 6 7 8 9 10
Generally, the book fulfills the general objectives of Teaching English language in Saudi Arabia. Generally, the book fulfills the general objectives of teaching English language for elementary stage. The terminal objectives specified explicitly in the pupil's book are based on some theoretical background. The terminal objectives define the desired degree of mastery. The terminal objectives meet the needs and wants of sixth graders. The developmental objectives are specified at the beginning of each lesson in the teacher's book. They are clear and precise. They are measurable. They suit the level of sixth graders. They contribute to the attainment of terminal objectives. E- Topic Contents
1 2 3 4
The topics of the book are varied and engaging to appeal to sixth graders with different interests and personalities. The topics encourage students to express their own views. The book avoids potentially embarrassing or disturbing topics. The topics allow students to think critically. F- Language Contents
1 2 3 4
The book covers the main grammar items appropriate to sixth graders. The book includes adequate materials for teaching vocabulary. The book includes adequate material for pronunciation work. The materials for teaching grammar, vocabulary, and pronunciation are graded in an appropriate manner. G- Social and Cultural Contexts
1
The social and cultural contexts in the book are comprehensible.
2
The content of the book is free from stereotypical images and information.
3
Students can learn about the inner lives of the characters used in the book.
4
The book expresses positive views of ethnic origins, occupations, age groups, social groups and disability.
107
H- Language Skills 1 2 3 4 5 6
The four skills are adequately covered. There is material for integrated skills work Listening material is well recorded, as authentic as possible, and accompanied by background information, questions and activities. There is sufficient reading material. (There is a range of varied and interesting reading text that can engage students cognitively and effectively.) There is sufficient material for spoken English ( e.g. dialogues, role-plays, etc.) Writing activities are suitable in terms of length, degree of accuracy, and amount of guidance. I- Teachability
1 2 3
The book helps teachers to minimize their preparation time. The book helps teachers exploit the activities to meet the students' expectations. The book helps teachers cater for mixed ability students and classes of different sizes. J- Flexibility
1 2 3 4
The book is appealing and useful to the students. The book caters for different levels of formality. The book provides opportunity for teachers and students to localize activities. The book caters for different preferred learning styles. K- Teaching Methods
1 2 3 4
The teaching methods used in the book are the latest in the field. The methods used are student-centered. The methods used allow students to talk more than teachers. The methods used allow various class activities. L- Practice and Testing
1 2 3 4 5 6 7
The book provides a variety of meaningful and mechanical exercises and activities to practice language items and skills. It provides communicative exercises and activities that help students carry out their communicative tasks in real life. Every exercise has a clear direction. There is a reasonable and appropriate number of exercises. The tests are valid and contain correct language. The book provides periodical revisions for diagnostic purposes. The book provides models for final achievement tests.
Thank you very much for cooperation
108 Appendix B The Jury Committee of the Questionnaire
1. Dr. Ghalib Raba'ba Ph.D in Educational Linguistics Department of English King Saud University
2. Abdulrahman Al-Tammami Chairman of the English Language Department Riyadh Directorate of Education MA in Applied Linguistics
3. Saleh Al-Qahtani Academic Supervisor English Language Institute - Riyadh Airbase MA in Applied Linguistics
4. Timothy Moloney English Language Teacher English Language Institute Riyadh Airbase Master in TEFL
5. Robert Lindsay English Language Teacher English Language Institute Riyadh Airbase Master of Education
