The Usability of SEEQ in Quality Evaluation of Arabic Secondary Education in Malaysia

Evaluation of the quality of Arabic education in Malaysia needs to be conducted on a continuous basis to achieve the objectives of Arabic education. Student Evaluation of Education Quality (SEEQ) was proposed as one of the evaluation instruments of the quality of Arabic education. This study aimed to evaluate the usability of SEEQ in the context of Malaysian education, including its validity and reliability based on the collected local data. In this study, a total of 491 samples from Islamic National Secondary Schools (SMK(A)) throughout Malaysia were involved. Confirmatory Factor Analysis (CFA) and Cronbach’s alpha for internal consistency were performed using AMOS and SPSS.A new model of SEEQ (8 constructs and 25 of 31 items were retained after modifications) with good readings off it indices and alpha value was obtained. This study suggests using the new model of SEEQ as the indicator of the quality of Arabic education in Malaysia.


Introduction
The history of Arabic education in Malaysia began with the arrival of Islam in this region.The residents of Tanah Melayu had first recognised the Arabic letters and its enunciation through Islamic religious practices that was related to Arabic.However, during that time, Arabic was not intensely taught and studied since the focus was given further to the teaching of Al-Quran and Fardhu Ain (Khoo, 1980).As such, Arabic was taught per se through religious studies that using books written in Arabic (Rejab, 1992).
It can be said that the formal Arabic studies in Malaysia began with the emergence of hut schools in the 18th century (Jusuh, 1990).Initially, the study focused on the study of syntax and morphology that served as a pre-requirement for the study of Islamic knowledge learned afterward (Samah, 2009).In 1996, through the Education Act 1996, Arabic was officially included in the Malaysian educational curriculum as a foreign language subject and was taught in schools as one of the linguistic subjects.Today, Arabic is not only taught in Arabic stream schools anymore since it has also been made available as an elective subject in boarding schools and higher education institutions.The learning and teaching of Arabic were further expanded with the establishment of the departments of Arabic language in public higher education institutions.Universiti Kebangsaan Malaysia was the first higher learning institution to establish the Department of Arabic Studies and Islamic Civilisation, followed by International Islamic University Malaysia through the Department of Arabic Language, and then University of Malaya through the Department of Arabic Language of the Faculty of Language and Linguistics.Universiti Putra Malaysia has also established its Arabic Language Unit and subsequently in 2005, Universiti Sains Islam Malaysia established its Arabic Language and Communication Programme in its Faculty of Major Language Studies.The last one was the same establishment in Universiti Sultan Zainal Abidin (UniSZA).
These favourable situations had placed Arabic in a special situation.Asma Omar (in Mezah, 2009) asserted that it is difficult to treat Arabic as a foreign language because of its status in Islamic society in Malaysia.Therefore, Arabic has its own status compared to other foreign languages based on the observational history of Islamic studies, culture and national education policy.

Problem Statement
Despite the robust growth of teaching and learning Arabic in all levels from primary education to university level throughout Malaysia, there are a few shortcomings which have become an open secret among the researchers of Arabic education and has also been acknowledged by the authorities.This weakness often to be a topic of discussion in academic research and educational seminars (Mezah, 2009;Rahimi, Ghani, & Abdullah, 2008).For example, studies done by Abdullah (2009), Mezah (2009) and Jacob ( 2007) on the quality of Arabic essay writing in the secondary and university levels had found that the levels of student performance were less than satisfactory and need attention.The weaknesses were particularly in terms of vocabulary such as inadequate vocabulary, inappropriate choice of words in sentence context, misspellings, word order not following the right methods, redundant use of words, convoluted sentences and a lot more.This study, among others things, revealed that the rooted cause is the abandonment of morphology aspects from both teachers and students when teaching and learning Arabic.Moreover, the attitude, enthusiasm and initiative factors of the Arabic students themselves had also contributed to this drawback.
Language is a skill.In this case, the Arabic's fundamental aspects that should be mastered by the students have also suffered a similar problem from its mastery aspect.A study done by Rahimi et al. (2006) among the religious secondary school students of Yayasan Islam Kelantan had described this situation.A total of 1665 form two students from 16 religious secondary schools throughout Kelantan had been selected as respondents.The study was conducted on five aspects of listening skill; (i) sub-skill of listening comprehension of the Arabic enunciation; (ii) sub-skill of listening comprehension of the lexical meaning of words; (iii) sub-skill of listening comprehension of the contextual meaning of words; (iv) sub-skill of listening comprehension of literal texts; and (v) sub-skill of listening comprehension of inferential texts.Findings of the study showed that the first sub-skill was at a high level; the second and third sub-skills were average; and the fourth and fifth sub-skills were at poor levels.These data showed that the listening level was at a danger level.Rahimi et al. (2006) also stressed that these weaknesses had indirectly affected the other skills based on the study done on speaking skills by Nik Rahimi in year 2000.This condition is following the basis that listening skill is regarded as the first level of the four language skills in learning taxonomy.
Based on the external and internal weaknesses of the Arabic students, these studies and others had provided early indicators of the needs for revaluation of the Arabic education ecosystem especially in secondary schools (see, Mohamad, 2009;Teh et al., 2010;Rahman & Ahman, 2007;Jabar, 2004;Ismail & Samah, 2007;Samah & Bakar, 2008;Samah & Ismail, 2006).Compared to other levels of Arabic education, the secondary level obtained numerous research focus that revealed these weaknesses.The evaluation and quality assurance process for the performance of the education quality is one of the important efforts implemented in each education system throughout the world.Malaysia, under the supervision of the Ministry of Education has started the process of evaluation and quality control of the Arabic education with various public initiatives such as the establishment of congregation unit of inspectors and quality assurance, the standard formation of Malaysian teachers, the appointment of excellent teachers based on subjects, regular teaching supervisions and others.
Evaluation instrument is one of the efforts to improve and control the quality of education.Instruments become the popular choice among educational researches especially with the students' participation as evaluators.This field is known as Student Evaluation of Teaching (SET).Student Evaluation of Education Quality (SEEQ) that was developed by Marsh (1982) is one of the most popular instruments of SET in educational world which has been implemented in a few countries.SEEQ is developed with the consideration of multiple dimensions that are present in teaching and learning, thus making it suitable to be used for all educational subjects.Therefore, it is widely utilised all over the world and considered to be one of the most reliable and suitable instruments to measure the effectiveness of teaching.The experts' recognition of the SEEQ questionnaire and the comprehensiveness of its measuring parameters encourage researchers to also implement it in Malaysia.The use of SEEQ in the context of Arabic education is expected to help in the process of evaluation and quality control with regards to other evaluation method by the other parties.However, the development of an instrument will not stop in the development phase, but continues throughout all phases, places and time.The existing instrument such as SEEQ needs to go through the psychometrics measurement evaluation process such as reliability, validity and quality of the items for it to remain relevant.These tests are more likely in the event of relocation, changes in respondents and time of study compared to the original condition of the instrument (Cohen, Manion, & Morrison, 2013).

Student Evaluation of Education Quality (SEEQ)
Student Evaluation of Education Quality (SEEQ) is one of the standard instruments that takes into account the various dimensions in the Student Evalution of Teaching (SET) field.Marsh (1982) developed SEEQ in multi-dimensional form and used it for the first time in the United States of America.Since then, SEEQ has been widely used around the world and is considered as one of the most reliable and suitable instruments for measuring the effectiveness of teaching (Coffey & Gibbs, 2001).For instance, Marsh (1982) had reported the internal consistency of SEEQ scale in the United States as between 0.88 to 0.97.In the Australian context, the internal consistency of SEEQ was 0.89 (Marsh & Roche, 1993).Meanwhile, Al-Hinai (2011) in his study in Oman had found the Cronbach's alpha value of 0.87.Moreover, SEEQ had been cross-cultural and its usage was practised and proposed in various countries such as England (Coffey & Gibbs, 2001), Spain (Marsh, Tourón, & Wheeler, 1985), Thailand (Jarernvongrayab, 2007), China (Marsh, Hau, Chung, & Siu, 1997, 1998), Papua New Guinea, India, Nigeria, Flipina, Hong Kong and many more (Perry & Smart, 2007).According to Perry and Smart (2007), the study that used SEEQ across the border of the United States had supported the usability factor contained in SEEQ outside its original context.Apart from being able to be implemented across nations, SEEQ has also been proven to be able to be implemented across various educational subjects including foreign languages.The suitability of the application of SEEQ in the teaching of a foreign language was proven in the Al-Hinai ( 2011) study.This cross-subject implementation capability is one of the selection criteria rendering its use in the assessment of the teaching of Arabic as a foreign language in Malaysia.
Thus, the experts' recognition of the SEEQ questionnaire as well as its suitability to the educational assessment has led the researchers to use it as the measurement and quality control tool for the Arabic education in Malaysia.The use of the existing questionnaire is the chosen method as it is the result of expert's work compared to the self-developed questionnaire that could lead to various problems (Idris, 2010).This study is the first to use SEEQ in the context of education in Malaysia.
As other questionnaires in the Student Evaluation of Teaching (SET), SEEQ uses various dimensions and aspects to evaluate teaching.There are 9 dimensions that are taken into account, including the overall evaluation of the teacher and the subject.Multidimensionality attributed to the educational field or the social sciences that in general dealing with complex constructs.The dimensions in SEEQ are; learning value, teacher enthusiasm, organisation, group interaction, individual rapport, breadth of coverage, assessment, assignments and overall rating.All dimensions are measured by a five-point Likert scale which are; 1-very poor, 2-poor, 3-moderate, 4-good, 5-very good.
Dimension 1: Learning Value (4 items): This scale consists of four items that measure whether the students found the classes stimulating their mind and whether their interest in the subject increased or not.
Dimension 2: Teacher Enthusiasm (4 items): This scale asks the students to evaluate their teachers' morale and whether he or she is able to attract students' attention with his or her method of teaching.Dimension 3: Organisation (4 items): This scale asks whether the course materials are prepared properly and presented clearly, and whether the objectives of the course are met accordingly.

Dimension 4: Group Interaction (4 items):
This scale asks the students to evaluate their teachers' ability to encourage students to participate in class discussions and to express their own ideas, as well as to encourage them to get further assistance from teachers.
Dimension 5: Individual Rapport (4 items): It evaluates the ability of a teacher to provide some spaces to the students by taking into account individual differences among students and his or her readiness to provide help and supports to the students.Dimension 6: Breadth Of Coverage (4 items): It measures whether the teacher discusses the various views and whether he or she is comparing the various theories in his or her teaching.Dimension 9: Overall Rating (2 items): This is the overall rating of the items.One of the items asks about the teacher's rating in overall, while the other item asks about the subject's overall rating.
The original version of SEEQ contains the scale of the 10th category other than the earlier 9 dimensions which is the criteria of the subjects/students.However, it is an additional scale (subscale) of the subject's or the student's background factor and it is not considered to be included in the evaluation of education (Al-Hinai, 2011).Therefore, this analysis does not include the 10th category as it is out of the context of this study.Moreover, the scale was formed specifically for the subjects in higher education, not for the secondary level which is the focus of this study.

Research Methodology
Problem statement has mentioned the selection of SEEQ as the instrument for evaluation and control of the Arabic Language education in Malaysia.Thus, the purpose of this research is to evaluate the use of SEEQ in the context of Malaysian education.This research covers the validity and reliability of SEEQ based upon the collected local data.Hence, the Confirmatory Factor Analysis (CFA) is implemented.Harrington (2009) suggested that'…CFA can be used to examine whether the original structure of the measure works well in the new population'.In this research, CFA is used with the assistance of Analysis Of Moment Structure (AMOS) version 18.To test the internal consistency, Cronbach's alpha is used for various scaled instruments such as SEEQ by using SPSS software.The evaluation of the usability of SEEQ in CFA is implemented based on fit indices such as CMIN/df, CFI, GFI, AGFI and RMSEA readings.This study used the following analytical readings as proposed by Hair, Black, Babin, and Anderson (2010) A total of 511 respondents from national religious secondary school (SMKA) all around Malaysia have participated in this research.The respondents in this study were chosen randomly from 13 schools that were selected based on cluster sampling.Cluster Sampling was an ideal sampling method in order to ensure that the samples fully represented the whole research population across Malaysia.Zones or states were used as the basis for determining the study's location sampling.However, only 491 respondents retained after data cleaning is done.According to Kline(2005), a sample that exceed 200 may be acceptable to most of the measurement model.

Findings
The first hierarchy in the Confirmatory Factor Analysis (CFA) is applied to the SEEQ model using AMOS.A total of 31 items which represent 8 constructs was drawn on AMOS canvas with the expected correlation between the 8 constructs.3 and 4. The internal consistency reliability test on the final model is done using the Cronbach's alpha analysis.According to Idris (2010), a complex construct uses various measuring items to measure the same construct.Hence, the internal reliability of all the items were measured as all items have the same role.After the final model is acquired, the data is added into SPSS for Cronbach's alpha analysis.As for the internal consistency test, Cronbach's alpha is used for various scaled instruments such as SEEQ and Semantic Differential or Kuder-Richardson for instrument with dichotomy item.Researchers have done separate analysis for each construct and an overall analysis as done by al-Hinai(2011).The result of the analysis is as shown in Table 6 Findings of Cronbach's alpha's Internal Consistency.Overall Cronbach's alpha analysis shows that the instrument is at a high internal consistency which is 0.939.The value obtained is almost as similar as the one obtained by Marsh (1982), Marsh and Roche (1993) and Al-Hinai (2011).On the other hand, Cronbach's alpha analysis which is done separately for each construct shows quite a different value than the overall value.These differences are the result of the Cronbach's alpha index that is influenced by number of items in a single scale.Therefore, Pallant (2011) suggested that the 0.7 alpha index value or higher is better for a scaled instrument that has ten or more items.On the other hand, a 0.5 alpha value is good for a less than ten items scaled instrument.The alpha value of each construct in this instrument is considered as very good since they are higher than 0.735 with only 2 to 4 items.

Conclusion
The final SEEQ model retains 8 original construct i.e.; learning value, teacher enthusiasm, organisation, group interaction, individual rapport, breadth of coverage, assessment and assignments.A total of 25 items were found to be measuring well the 8 constructs with the Factor Loading exceeds 0.60.The analysis of the fit indices shows a good construct validity and stability.In addition, the reliability of SEEQ from the internal consistency aspect is high from 0.735 to 0.828 and the overall alpha value is 0.939.The overall findings support the use of the final SEEQ model for future evaluation of the quality of the Arabic Language education in Malaysia that uses the same or almost similar samples.It needs to be reminded that the final model is resulted from the exploratory nature of modification rather than confirmation.Therefore, it is suggested that the final SEEQ model is to be tested again for consolidation (Harrington, 2009).Researchers expect with the continuation of this research or suchlike will help to improve the quality of the Arabic Language education in Malaysia.

Dimension 7 :
Assessment (3 items):This scale assesses the quality, equality, values and appropriateness of the teacher's feedbacks and tests on the subject's objectives.Dimension 8: Assignment (2 items): This scale asks the students to give values to the texts and additional readings provided by the teacher, and to evaluate the contribution of the tasks in the appreciation and understanding of the subject.
Fit indices of the first CFA shows that; CMIN/df = 2.589, CFI = 0.916, GFI = 0.874, AGFI = 0.846 and RMSEA = 0.057.Diagram 1 shows the SEEQ model at the primary stage.According to the suggested reading analysis by Hair et al. (2010), the initial model does not show a perfect fit with Malaysian data.Therefore, 6 modification steps were performed to acquire appropriate index reading.These modification steps are based on the recommendation of Hair et al. (2010)namely, (1) examine the Factor Loading of each item, (2) examine Standardised Residual, and (3) examine the modification index.Diagram 1. CFA-SEEQ primary stage measurement model The 6 modification steps have removed 2 items which are question 1 and question 20 due to low Factor Loading.Question 1 = 0.37 and question 20 = 0.58.Since SEEQ is a well-known instrument, the level of the Factor Loading should exceed 0.60 (Zainudin Awang, 2011).The low level of the Factor Loading symbolizes the weakness of the representative items against a construct in the model.Throughout the modification steps, there are 4 other question items removed due to high correlation between the four pairs of items.These pairs of items are; 15e16, e6e7, e413 and e8e11.These pairs of items have the highest correlation value among other items.The high value shows that there is a repetition or overlap (redundant) items in measuring the construct.Therefore, four items from each pair were removed based on the lowest Factor Loading.All steps were performed based on improvement in the fit indices reading.Refer to Tables

Table 3 .
Summary of the modification steps and the fit indices improvement

Table 4 .
Summary of the items removed throughout the modification

Table 5 .
Findings of Cronbach's Alpha's internal consistency