Relooking at the ESL Reading Comprehension Assessment for Malaysian Primary Schools

The purpose of the study is to have a relook at the ESL reading comprehension assessment system for Malaysian Year Five students. Traditionally, the ESL teachers have been assessing and reporting on their primary year’s students by merely giving a composite grade with some vague remarks. This process has been used and is still being employed in spite of the numerous advances and progress that have been made in the realm of education. To gauge the students’ reading ability there is a need to take a serious look into the way teachers assess the students. In this ESL reading comprehension assessment system, a set of standardised generic reading comprehension test, a reading matrix and reading performance descriptors were developed. The findings revealed that Year Five respondents at reading performance Band 1, Band 2, Band 3, Band 4 and Band 5 have acquired the literal, reorganization and inferential reading sub-skills to a certain extent. The results obtained were found to be consistent indicating that the ESL reading assessment is reliable and valid to a large extent as revealed by a second administration of the test conducted in a few other selected primary schools. This ESL reading comprehension system can provide information on students’ reading ability at both the micro and macro levels. At the micro level, ESL teachers can plan their teaching instructions that tailor to the needs of the students. At the macro level, it can assist the district as well as the state education departments in Malaysia to plan reading programmes for primary school students.


Introduction
The process of assessing reading comprehension has largely remained unchanged over the years in spite of the transformations that have taken place due to changes advocated by policymakers or trends in the world of education.Assessments are still used to evaluate the performances of students whether it is norm referenced or criterion referenced, to inform instructions, to find out whether the students can gain access into the appropriate programs and even to evaluate the program.In the current scenario within the Malaysian context, the assessment of reading comprehension is not informative enough to help the teachers to make informed decisions.Though the teachers might be genuine in their attempts to help their students to perform and succeed and become good readers they are actually in a very helpless state as the information derived from the reading comprehension assessments was unable to provide sufficient data or information to help the students individually or as a class or even for the whole school.Thus, as Pearson and Hamm (2003), rightly puts it, "we need better assessments so that we can respond to the pleas of teachers desperate for useful tools to assist them in meeting individual needs".
Reading comprehension is a cognitive process that takes place when an individual interacts with the text.It is disadvantageous for individuals who possess poor reading ability.In classroom teaching and learning, assessment is a crucial ongoing process as it enables teachers to identify what a learner can and cannot perform (Brown, 2004;Popham, 1999).Masters (2014) further reiterates that the information obtained through assessment helps to improve learning outcomes.The ultimate goal of assessment is not for making comparative judgement.Instead, it is to provide feedback to students by identifying their strengths and weaknesses.
The English language syllabus as specified by the Malaysian Ministry of Education (2003), has a very noble aim that is to equip the primary school students with the fundamental of language skills.The ESL students should be taught to read and understand texts holistically.To further enhance the development of the students' language abilities, the School Based Assessment was introduced and implemented by the Malaysian Ministry of Education in 2014.One of the developments accorded by the Malaysian Standard Curriculum Document and Assessment (2014) (also known as Dokumen Standard Kurikulum dan Pentaksiran, 2014), states that primary school students should be furnished with the ability to read for information independently by the end of Year 6.
English is taught as a second language in all Malaysian public schools.Primary schools students are taught reading comprehension in the ESL classroom.Upper primary students' English language performance is assessed in line with the format of Primary School Achievement Test (also known as Ujian Penilaian Sekolah Rendah).This test assesses not only students' reading comprehension but also vocabulary, language functions, grammar, sentence construction and note expansion.There is no detailed result given to the test other than a grade assigned to summarise the students' performances.The achievements from the results obtained are also utilized to evaluate the effectiveness of the ESL teaching (Mohd Sofi Ali, 2003) by the Malaysian Ministry of Education.Thus, there is no serious or in depth measurement of a student's reading comprehension per se.

Literature Review
In order to provide a fundamental concept for the present study, the ideas of reading comprehension, reading matrix, standardised reading comprehension test (SRCT) and conceptual framework are discussed in relation to the development of ESL reading comprehension assessment system for Year Five students.

Conceptualising Reading Comprehension
Reading is an essential skill to master for ESL and EFL students.With good reading ability, readers can achieve great academic development (Puteri Rohani Megat Abdul Rahim et al., 2017).The reading comprehension process which is basically defined as the interaction with a text involving a wide range of cognitive skills and processes (Woolley, 2011) is conceptualized as a skill that is analogous to the reading performance.Numerous researches and studies have revealed that children come to a particular class with different levels of reading abilities.In a case study by Ankrum and Bean (2008), they state that "children have always come to school with a range of literacy experiences and abilities and teachers have struggled for years to meet the needs of all of their learners".This is further supported by Santhi (2011) that each classroom consists of mixed-ability students; students possess different skills and progress at different rates.According to Charanjit Kaur Swaran et al. (2017), in Malaysia, the teachers are shifting their focus from the practice of assessment of learning to assessment for learning with the aim to improve students' ability.The test results of a single form of examination might not be able to reflect the multiple intelligences of students that students possess.Undeniably, the results obtained from the assessment do not reveal the true abilities of these students as the students are normally only given a composite grade.In Malaysia, the Primary School Achievement Test also assigns grades to basically summarise the students' English language performance.These composite grades do not adequately describe the students' reading ability as the reading comprehension only constitutes a small section of the test.This discrepancy has been highlighted by Lin et al. (2016), as they believe that grades on its own offer very limited information on learners' levels of reading ability.Two learners might obtain the same grade in a test but it cannot be claimed that both learners possess similar level of reading performance.
By and large, grades do not provide sufficient descriptions of students' strengths and weaknesses in detail.Undeniably, though the Malaysian Standard Curriculum Document and Assessment (2014) does provide teachers with descriptors of performance standard but the descriptors lacks details and do not guide the teachers to accurately identify the students' abilities.Consequently, the teachers are still left groping in the dark and the problem still persists with students not having information on their strengths and weaknesses in answering reading comprehension questions identified and described adequately and accurately.
There was another attempt by the Malaysian Ministry of Education in 2014 to address the issue of describing students' performance by adopting a performance scale to categorise schools into bands (Band 1 to Band 4).The school performing bands are based only on each particular school's overall performance on the various subjects that are taught in the schools and not specifically on reading comprehension abilities.Again, this effort is not of much help to the teachers.It only categorically identifies the school holistically and to take remedial action to improve its academic achievement.In order to resolve this problem, the researcher intends to relook into the assessment of reading comprehension and fill this vacuum by developing a reliable reading assessment system for primary schools in Malaysia.Ahlam Ali Salim Halali et al. (2017) highlighted that it is important for teachers to obtain information of students' learning process progressively so that teachers are be able to justify students' needs in a timely manner thus maintain or keep their instruction ongoing with students' development.Besides providing detailed information on students' reading ability to ESL teachers, the researcher also intends to develop a reliable reading assessment system that can solve the teachers' problems in developing reading comprehension tests.

Conceptualising the Reading Matrix
The Reading Matrix is conceptualised as a chart that indicates learners' reading abilities at a particular educational level Abdul Rashid et al. (2010) and Lin et al., (2016).The notion of reading ability is analogous to the Body Mass Index (BMI) chart.The BMI is based on your height and weight.It is one way to see if you are at a healthy weight.You are underweight when your BMI is less than 18.5.You are of healthy weight if your BMI is 18.5 to 24 while you are overweight when your BMI is 25 to 29.9.You are obese if your BMI is 30 or higher.IN order to measure your BMI, your height will be used to indicate whether you fall into any of the above categories.
Similarly, we can conceptually match the learners' test scores with the reading performance bands against the educational levels.As can be seen from the Reading Matrix below, the vertical column represents the educational levels of the students while the horizontal column represents the reading performance bands.For example, if a Year Five student scores marks within 29-36 marks, he/she would be classified under Band 2.

Standardised Reading Comprehension Test (SRCT)
To find out the BMI a weighing machine is needed in order to take the weight of the person being measured.Similarly, to be able to assess the reading performance of the students, the tool or instrument that is required would be the Standardised Reading Comprehension Test (SRCT).If the weighing machine is faulty, it will give a wrong measurement then the diagnosis could be totally irrelevant.The weighing machine will need to meet the specifications approved by the government body.This also applies to the SRCT which would have to piloted and tested for its practicality, validity and reliability.Briefly, the SRCT consists of comprehension questions at elementary, intermediate and advanced levels.The allocation of the difficulty of the test adopted the idea suggested by Mok (2000) where the test questions should practise wide enough difficulty distribution.This SRCT is based on Barrett's Taxonomy (1968) and Bloom's Taxonomy (2001) and is aligned to the Malaysian English Language Syllabus ( 2003) and Malaysian Standard Curriculum Document and Assessment (2014).

Research Design
This study involved four main stages to develop the ESL reading comprehension assessment system.It began with the development of a prototype reading comprehension test to determine if the instrument is able to elicit the desired information from the respondents (Postlethwaite, 2005).The test was then piloted to determine its reliability.The first administration of the standardised generic reading comprehension test, namely Set A was conducted in 8 selected primary schools.
At the second stage, a reading matrix was developed using a z-score that indicates how many standard deviations from the mean.Cut scores were used to categorise respondents into performance bands namely, Band 1, Band 2, Band 3, Band 4 and Band 5. Band 1 is the highest reading performance band whereas Band 5 is the lowest performance band.In short, the reading matrix is a chart that determines respondents' reading abilities through several bands.
At the third stage, reading performance descriptors were developed based on Malaysian English Language Syllabus ( 2003), and Malaysian Standard Curriculum Document and Assessment (2014).To enhance the practicality of the reading performance descriptors, the researchers also adapted the concept from British Columbia Performance Standards ( 2009), Delaware Comprehensive Assessment System (2010) and Minnesota Reading Assessment (2013).Qualitative data was also collected through a structured interview to provide ESL teachers a holistic picture on students' reading ability.
Finally, the ESL reading comprehension assessment system was tested to identify its reliability in terms of SRCT, reading matrix and reading performance descriptors.The researchers conducted the second administration of standardised generic reading comprehension test, namely Set B. A comparison was made in terms of the reliability coefficient of KR20 of Set A with Set B. The cut scores of reading matrix in Set A and Set B were compared.Another structured interview involving respondents in Set B was conducted to increase the reliability of the reading matrix.The reliability of reading performance descriptors was identified through the comparison of test scores obtained in Set A and Set B. Figure 1 summarises the 4 stages in developing the ESL reading comprehension assessment system Figure 1.The 4 stages in developing the ESL reading comprehension assessment system

Participants
The population of the study consists of 16 primary schools categorised as Band 1 schools, Band 2 schools, Band 3 schools and Band 4 schools from the district of Larut, Matang and Selama in Perak.According to the Malaysian Education Department, schools categorised as Band 1 refer to high performing schools whereas schools categorised as Band 4 refer to low performing schools.In this study, SRCT was administered on two different occasions, namely Set A and Set B as the researchers intended to identify and validate the reliability of the ESL reading comprehension assessment system.Both Set A and Set B respectively consisted of a school categorised as Band 1 school, three schools categorised as Band 2 schools, three schools categorised as Band 3 schools and a school categorised as Band 4 school.The respondents consisted of students from year 5 of all the schools specified above in accord with stratified random sampling.

Development of SRCT
A pilot study was conducted on the prototype reading comprehension test in a primary school.It involved respondents of Year 4, Year 5 and Year 6.The pilot study allowed the researchers to determine the validity and reliability of the test.The KR-20 value for the instrument was 0.80 which indicates that students' responses to the reading comprehension questions were consistent.Thus, the KR-20 value obtained in this study fulfilled the quality of being highly-reliable.
The standardised generic reading comprehension test consisted of three parts.Part 1 consisted of 12 reading comprehension questions at elementary level.Part 2 comprised 24 reading comprehension questions at intermediate level and Part 3 consisted of 14 reading comprehension questions at advanced level.
At each level, the comprehension questions consisted of literal, reorganisation and inferential comprehension questions.According to Day and Park (2005), literal comprehension questions require straightforward understanding of the text.Reorganisation comprehension questions require students to link information from different paragraphs for further understanding.Inferential comprehension questions demand students to combine their literal understanding with their prior knowledge as the answers are not stated explicitly in the text.

Development of Reading Matrix
Reading matrix is a chart that indicates learners' reading abilities at a particular educational level.In this study, the standardised generic reading comprehension test consisted of comprehension questions at elementary, intermediate and advanced levels.Logically, Year 4 students should be able to answer all comprehension questions at elementary level because they have completed the syllabus of Year 4. Year 5 students should be able to answer all comprehension questions at elementary and intermediate levels while Year 6 students should be able to answer all levels of comprehension questions.The Cut Scores for Each Reading Performance Band of the Reading Matrix was determined by using the test scores obtained from the pilot study to categorise the respondents into bands.To decide the range of cut scores between bands, z-scores were used.Z-score refers to the number of standard deviation from the mean (Carey, 2001).It is a measure of how many standard deviations are below or above the mean.If z-score is zero, the score is equal to the mean score.
In this study, the mean obtained from the pilot study was 20.The cut scores for the bands are summarised in Table 4.

Development of Reading Performance Descriptors
In this study, the respondents' reading performances were categorised into five bands (Band 1 to Band 5) based on Barrett's Taxonomy, Bloom's Taxonomy, Malaysian English Language Syllabus and Malaysian Standard Curriculum Document and Assessment.The percentages of respondents who managed to answer each sub-skill of reading comprehension question were analysed to determine the appropriate terms to describe the respondents' reading abilities.
To enhance the credibility of the reading descriptors, the researchers conducted structured interview to examine their capability and inability in answering reading comprehension questions.Table 5 is an excerpt of reading performance descriptors developed for Year Five students.
Reading descriptors serve as a set of diagnostic tool to identify learners' reading abilities (Abdul Rashid et al., 2010 andLin et al., 2016)

Testing the Reliability of the Reading Comprehension Assessment System
In order to test the reliability of the reading comprehension assessment system, a second administration of the standardised generic reading comprehension test was done, namely Set B. Similar to the procedure used in Set A to determine the test reliability, the reading comprehension test in Set B, was conducted in other eight selected schools which were also categorised as Band 1 schools, Band 2 schools, Band 3 schools and Band 4 schools in the district of Larut, Matang and Selama.From the test results obtained, the researcher intended to determine the test reliability from Set B to enable a comparison of the KR20 reliability coefficient between Set A and Set B to be made.
To determine the reliability of the reading matrix, the test scores obtained in Set B were analysed to identify the mean and standard deviation.The mean and standard deviation in both Set A and Set B were compared.In addition, with the mean and standard deviation obtained in Set B, the researchers also developed the cut scores for Set B. A comparison of cut scores between Set A and Set B was also done.To increase the credibility of the reading matrix, the researchers conducted a structured interview and the qualitative data was analysed in line with the learning standards as stated in the Malaysian Standard Curriculum Document and Assessment (2014) and Malaysian English Language Syllabus (2003).The descriptions of reading performance in each reading sub-skills stated in the afore-mentioned documents were referred to identify the respondents' reading performance.
Finally, the test scores obtained in Set B were compared with the test scores of Set A to check the reliability of the reading performance descriptors.

Findings
Results on the reliability of the ESL reading comprehension assessment system are summarised below.Set A refers to the first administration of the reading comprehension test, which was carried out to develop the cut scores, reading matrix and reading performance descriptors.To identify the reliability of the reading assessment system, a second administration of the reading comprehension test, namely Set B was conducted in eight other selected schools categorised as Band 1 schools, Band 2 schools, Band 3 schools and Band 4 schools.

Reliability of the Standardised Generic Reading Comprehension Test
Based on the findings, the reliability coefficient in Set B was identical to the reliability coefficient in Set A, that is 0.8.A conclusion could be drawn that the reading comprehension test is highly reliable.

Reliability of the Reading Matrix
To identify the reliability of the reading matrix, the test scores obtained in Set B were utilised to categorise the respondents into five bands (Band 1, Band 2, Band 3, Band 4 and Band 5).The mean and standard deviation in Set A and Set B were compared.The findings showed minor differences.Table 6 summarises the difference of mean and standard deviation in Set A and Set B. The researchers also compared the cut scores developed in Set A and Set B and the differences between the cut scores developed in Set A and Set B are illustrated in Table 7.

Verifying the Development of the Reading Performance Descriptors
The reading performance descriptors identified were utilised to account for the performances of the respondents who answered each sub-skill of reading comprehension questions in Set A and Set B. Band 1 indicates the lowest performing band while Band 5 represents the highest performing band.It can be safely concluded that the reading performance descriptors can be used to describe the Year Five students' reading performances adequately.
At the elementary level, both Band 1 students in Set A and Set B shared the same percentage (25.3%) in answering L2 comprehension questions (identifying main ideas).
At the intermediate level both sets of respondents' performance were close.Band 1 respondents in Set A and Set B showed very limited ability in answering R1 comprehension questions (read and understand the meanings of words by guessing their meaning through the contextual clues), that is, less than 30% of the respondents could answer the questions correctly.However, more than 70% of Band 5 respondents in both Set A and Set B could answer R1 comprehension questions correctly.
At the advanced level, it came as no surprise that Band 1 respondents in Set A and Set B exhibited very limited ability in answering L1 comprehension questions (identifying supporting details).While only 17.7% and 12.7% of the respondents from Set A and Set B respectively could answer the questions correctly.Furthermore, Band 1 and Band 2 respondents in both Set A and Set B were found to have similar performance in answering R2 comprehension questions, that is, 24.7% and 24.9% respectively.Band 3 respondents revealed an average performance in answering L1 and R2 comprehension questions.

Dicussions
The study presented highlights the needs in developing a reliable ESL reading comprehension assessment system for primary school students to provide specific information for ESL teachers, students and also parents.Given the fact that Malaysian students need improvement in reading (Inderjit, 2014) and Malaysian ESL teachers in primary schools only possess students' grades which do not describe students' reading abilities explicitly.The urgency and importance placed on insisting teachers be well equipped with relevant and accurate information on the students' reading performance is to allow appropriate actions to be taken by the teachers and relevant authorities to deal with the poor reading performance in schools.This need for the reading comprehension system is timely as we cannot produce different result with the same way of describing students' performances.
Relooking into how teachers can collect information about students' performances can go a long way in enabling the schools to achieve the aims of equipping primary school students with the fundamental language skills.
The standardised generic reading comprehension test consists of the best possible combination of reading comprehension questions developed to evaluate the students' reading performance.However, the test be reliable and valid.As emphasised by Gay and Airasian (2000), and reiterated by Caldwell (2014), it is only when a test is reliable, dependable and trustworthy, can we safely conclude that the scores of the test are consistent no matter how many times the students sit for the test.By categorising the students' reading performance into bands and matching the students' test scores to the reading matrix, the ESL teachers can refer to the reading performance descriptors to get a vivid picture about students' specific and detailed reading ability.
In the present system, primary school ESL teachers are provided with a standard curriculum document and assessment but the instrument for assessing students' reading ability is not supplied.As a result, ESL teachers had to resort to the reading comprehension texts from the workbooks and whatever materials they can lay their hands on.Unfortunately, most of the materials do not come with the descriptors for the reading skills or strategies that the teachers are trying to develop.In addition, even if these reading materials do make provision for the reading descriptors, they also do not tally with the descriptions as stated in the standard curriculum document and assessment.Thus, it might be difficult for ESL teachers to design teaching instructions accordingly as there is a lack of valid, reliable and practical instrument for reading.Another important issue is that the current formative assessment in classroom cannot provide further details other than just grades and marks.With little information about the students' reading abilities, ESL teachers can hardly track their students' strengths and weaknesses.
It is hoped that the development of ESL reading comprehension assessment system can assist ESL teachers in determining not only students' reading abilities accurately but adequately and conveniently as well.The ESL teachers are then relieved from the tedious task of obtaining the necessary reading comprehension test and only need to focus on planning the teaching instructions and their teaching.

Conclusion
The ESL reading comprehension assessment system is proven valid and reliable.Therefore, it can be safely used by ESL teachers in upper primary schools.This system helps to lessen teachers' job in creating the test thus leaving them more towards designing teaching instructions and assisting the students.This assessment intends to provide ESL primary school teachers a detailed information about what pupils can and cannot do.

Table 1 .
The reading matrix We find that the reading matrix can gauge students' reading abilities through several bands.The reading bands are supported by a set of clear and detailed reading descriptors on what the students have or have not acquired.For instance, Year 4 students should meet standard at Band 3. If students score more than 28 marks, they are categorised as above standard.Year 5 students should meet standard at Band 2. If the students score less than 29 marks, they are categorised as below standard.Year 5 students who are placed in Band 1 are categorised as above standard.Year 6 students should meet the standard in Band 1.If they score less than 37 marks, they are categorised as below standard.

Table 2 .
Reading comprehension questions at elementary, intermediate and advanced levels

Table 3 .
Reading matrixBy referring to Table3, the vertical column represents the educational levels of the respondents while the horizontal column represents the reading performance bands.For example, if a Year Five student scores marks within 29-36 marks, the student is classified under Band 2 category.

Table 4 .
Cut scores

Table 5 .
. They give teachers a vivid picture of what students can and cannot do.According to British Columbia Performance Standards (2009), reading performance standards offer information on students' ability to apply skills and strategies acquired.Such development aims to support ongoing instruction and assessment.In this study, the researchers adapted the concept of reading performance descriptors from British Columbia Performance Standards (2009), Minnesota Academic Standards in Reading (2013), and the Delaware Comprehensive Assessment System (2010).Sample of reading performance descriptors for year five respondents The reading performance descriptors were developed based on the test scores obtained from the first administration of the standardised generic reading comprehension test, namely Set A. The test administration involved eight selected schools categorised as Band 1 schools, Band 2 schools, Band 3 schools and Band 4 schools.The reading descriptors were developed in line with the Malaysian English language syllabus(2003)and the Malaysian Standard Curriculum Document and Assessment (2014).

Table 6 .
Mean and standard deviation in Set A and Set B

Table 7 .
Difference between the cut Scores developed in Set A and Set B

Table 8
illustrates the comparison of percentages of respondents who answered each sub-skill of reading correctly in Set A and Set B.

Table 8 .
Comparison of percentages of respondents who answered each sub-skill of comprehension question correctly in Set A and Set B