Pictorial, Textual, and Picto-Textual Glosses in E-Reading: A Comparative Study

This research explored the effects of three types of multimedia glosses on the reading comprehension of learners in an EFL context. From among the three experimental groups under study, one received treatment on five academic reading passages through picto-textual glosses where both textual definitions and relevant images of words popped up, thus helping learners surmise the meanings of the keywords. In a similar vein, the other two groups received treatment on the same passages, but used either pictorial or textual glosses where pertinent pictures or textual definitions of the keywords appeared on the screen. The experiment showed that the picto-textual-gloss group outperformed the other two groups, and that the pictorial-gloss group outperformed the textual-gloss group. The study thus underpinned Paivio’s (1971, 1986, 1991) Dual Coding Theory, arguing that when information is available in two modes of presentation, it becomes more elaborate and thus more memorable. When applied to reading comprehension, the idea suggests that the combination of textual and visual definitions might help learners better decipher the meanings of keywords in reading passages, and hence arrive at an even deeper comprehension of the texts.


Background
There is a general consensus that multimedia courseware is potentially useful for second language reading, as a given text can be rendered comprehensible to L2 learners through glosses or annotations (Davis, 1989;Martinez-Lage, 1997;Roby, 1999). The term "gloss" refers to an explanation or a definition of words written on margins in reading passages that helps learners readily work out the meanings of the words (Shiki, 2008). Few studies (Chun & Plass, 1996;Lomicka, 1998;Kost, Foss, & Lenzini, 1999;Al-Seghayer, 2001;Jones & Plass, 2002;Rott & Williams, 2003;Bowles, 2004;Ko, 2005;Yoshii, 2006;Akbulut, 2008) have corroborated the contributions of multimedia glosses to reading comprehension, in general, and vocabulary acquisition, in particular. Chun & Plass (1996), for instance, examined the effects of multimedia glosses on reading comprehension. They conducted three studies involving 160 students of German using CyberBuch, a multimedia software application. The results of these experiments revealed that multimedia glosses significantly aided in comprehension, and that vocabulary annotations consisting of both visual and verbal definitions were more effective than the ones providing textual definitions alone. Likewise, Lomicka (1998) explored the way multimedia annotations might have affected the level of comprehension. The participants were 12 college students in a second-semester French course. Learners were asked to read a text under one of three conditions: full-gloss condition, limited-gloss condition, and no-gloss condition. The experiment showed that computerized reading with full glossing led to a deeper understanding of the reading passage. In Ko's (2005) study, in a similar vein, 106 undergraduates at a university in Korea were asked to go through the reading passages under one of three conditions: no gloss, Korean gloss, and English gloss. After reading, they were asked to take a multiple-choice reading comprehension test and answer a questionnaire. The experiment showed that both types of glosses made the learners' reading comprehension smoother and faster as compared with the situation where the students read the narratives without glosses. Similarly, Akbulut (2008) conducted an experiment where 54 seniors were provided with a hypertext and were told to write down everything they remembered at the end of reading. The reading passage was annotated through a variety of media, including texts, pictures, audio, etc. where glosses would provide information both at the word level (definitions of words) and the topic level (extra information about the topic). The experiment revealed that the hypermedia environment drawing on glosses significantly improved comprehension of the reading text.
The aforementioned few studies have thus favored the use of multimedia glossing as an effective pedagogical technique of improving vocabulary acquisition and reading comprehension. Nevertheless, most of such experiments, including Al-Seghayer's (2001) study, have suffered from a small sample size, short period of experiment, lack of multimedia courseware with a user-friendly interface, and most importantly, a criterion to which the efficacy of glossing could have been appraised. Furthermore, the main focus in such studies was on vocabulary acquisition rather than reading comprehension. The main instrument in such experiments used for data collection was a vocabulary post-test measuring the degree to which learners had mastered the annotated vocabulary items. The number of comprehension questions was limited; hence, learners' scores on the comprehension tests would not have yielded an accurate estimate of their degree of comprehension. Moreover, the researchers were not specific about the way their programs had controlled the amount of treatment across the groups under study. Some participants might have received more treatment, and therefore their performance might have confounded the effects of glossing. These limitations, accordingly, should be taken into consideration in the follow-up studies so that we can make more accurate judgments about the degree to which glosses in multimedia settings might prove fruitful in expanding learners' vocabulary repertoire and improving their reading comprehension.

The Present Study
Regrettably, a great many students in EFL environments including Iran fail to harness their reading skills in order to readily acquire knowledge disseminated through scholarly textbooks. This is of utmost significance for learners pursuing their studies in academic settings where they are required to go through hundreds of pages of academic texts in a short span of time. Following the traditional methods of improving learners' overall comprehension, most teachers recommend the use of the famous "guessing technique" to help learners surmise the meanings of keywords in reading passages through their neighboring words. Yet, in case learners have difficulty guessing the meanings of the words, they should repeatedly consult their dictionaries, which may have an adverse effect on their concentration. Hence, developing effective pedagogical methods to help alleviate this problem seems mandatory. Convictions are strong that multimedia environments have the potential to sustain learners' concentration during reading by annotating keywords with a single click. Marginal glosses in a variety of media can be accessible to students by clicking on appropriate buttons or by simply moving the mouse cursor over keywords. This can help learners sustain their attention during reading that might, then, improve their overall understanding of reading passages. The underlying assumption is that the less the interruption caused by students' inability to process connected texts, largely owing to their limited vocabulary repertoire and their frequent consultation with the dictionary, the better their concentration and accordingly, the better their comprehension of texts.
Given a paucity of research on the contribution of multimedia glosses to reading comprehension and due to the limitations the aforementioned studies faced, the present research sought to provide grounds for more accurate judgments to be made about the efficacy of glossing in multimedia environments by using the multimedia programs featuring an interactive, user-friendly interface where learners could have readily interacted with the computer, thus minimizing the adverse effect their unfamiliarity with using the computer might have had on their performance, by choosing a much bigger sample size, ensuring generalizability, and by providing a criterion to which the likely effects of glosses on subjects' comprehension could have been assessed. Moreover, the built-in countdown timer could have further minimized the detrimental effects of time and amount of treatment, as moderating variables, by controlling the amount of time spent on the reading passages across the groups under study. It was hypothesized that in case these variables had remained uncontrolled, the groups under study would have received varying amounts of treatment and their performance on the post-test would have been affected accordingly.

Purpose of the Study
This study aimed to explore the effects of three types of multimedia glosses, i.e., textual, pictorial, and picto-textual, on the reading comprehension of Iranian intermediate-level EFL learners.

Research Question and Hypothesis
This research sought to find an empirically justified answer to the following question: Q: Is there any significant difference in the use of the multimedia program drawing on textual glosses, the one using pictorial glosses, and the one using a combination of textual and pictorial glosses in aiding in EFL learners' reading comprehension?
A pertinent null hypothesis was formulated as follows: : There is no statistically significant difference in the use of the multimedia program drawing on textual glosses, the one using pictorial glosses, and the one using a combination of textual and pictorial glosses in aiding in EFL Learners' reading comprehension.

Subjects
The subjects involved 120 students who were majoring in Teaching English as a Foreign Language (TEFL) at Islamic Azad University-Rasht Branch, Iran. These subjects were identified as intermediate-level EFL learners based on their scores on the proficiency test.

Instruments
The main instruments under study comprised there types of Multimedia Computer-Assisted Language Learning (MCALL) programs authored by one of the researchers. One type, known as Pictorial-Gloss MCALL Courseware, consisted of five academic reading passages whose keywords were annotated pictorially, i.e., definitions (glosses) of words whose meanings were key to the understanding of the passages were given to the subjects through static pictures. The second type, known as Textual-Gloss MCALL Courseware, comprised the same reading passages, but used textual definitions only. Likewise, the third type, known as Picto-Textual-Gloss MCALL Courseware, used a combination of both textual and pictorial definitions to help make meanings clear to the subjects. The programs also came with a built-in countdown timer, allowing the researchers to control the time variable and the amount of treatment.
Other instruments involved an English proficiency test to choose subjects of the desired level of language proficiency and a randomizer to randomly assign the participants to equivalent groups of subjects.

Procedure
At the beginning of the experiment, a proficiency test of receptive skills based on the UCLES IELTS examination papers was administered to 286 students who were majoring in Teaching English as a Foreign Language (TEFL) at Islamic Azad University-Rasht Branch, Iran. From among them, 120 students who got five on the test were identified as intermediate-level learners and were randomly assigned to four equivalent groups of subjects: one pilot group and three experimental groups. In order to randomly assign the participants to the groups under study, the researchers drew on a randomizer called SuperCool Random Number Generator featuring the capability of generating random sets of numbers from within a range. Each participant was given a number from 1 to 120. The program then randomized the numbers in such a way that the first 30 subjects were put in the pilot group and the second, the third, and the fourth 30 subjects were assigned to the experimental groups respectively. Accordingly, all the participants had an equal chance of being put into any one of the four groups under study. It should be noted, however, that the subjects comprised mixed groups of males and females.
Next, a multimedia pre-test comprising five academic reading passages centering on topics, such as Insect's Anatomy, Supernovae, Deep Vein Thrombosis (DVT), etc. was first administered to the pilot group under study to be standardized. The test consisted of 50 comprehension questions in the multiple-choice format. Each item correctly answered would receive a score of one mark, and the total score possible would be 50.
Once the students took the test, their papers were scored and the test item statistics were then calculated through an item analyzer called Test Analysis Program (TAP). This program, developed by Brooks (2003) at Ohio University (http://oak.cats.ohiou.edu/~brooksg/downloads/tap.exe), features the capability of marking defective items with an asterisk. The analysis showed that the Item Facility indexes of all the items on the test had not exceeded 0.63. This was desirable, as this implied that the questions were not too easy so that a great many students would have been able to get them right without using glosses.
The next step involved computing the reliability index of the test through a Cronbach's Alpha. It turned out to be 0.94, which was significant. Next, to establish the test construct validity, the researchers used SPSS to do a factor analysis. Using the "Principal Component Extraction" technique, the researchers then determined the number of hypothetical factors involved, as well as the degree to which the items on the test correlated with the latent construct. This technique assumes that there are as many theoretical factors as there are items on the test. Moreover, it assumes that each factor or construct has an "eigenvalue" that represents the amount of test variance accounted for by the very construct. Following these assumptions, the program then extracted all those hypothetical factors whose eigenvalues were above unity. The analysis result revealed that only one factor was extracted, as only one had an eigenvalue well above unity (71.24). This was desirable, as this showed that only one factor contributed significantly to the total variance of the test. In other words, all the items highly correlated with a single construct, i.e., the reading ability.
Once the construct validity of the test was established, it was administered to the three target groups under study. The purpose of pre-testing was twofold: to ascertain the homogeneity of the groups a priori regarding the passages being introduced and to establish a criterion to which the efficacy of glosses could have been appraised. At the researchers' signal, all the subjects in the three groups sat at the computer terminals in the University's lab and wore headsets. Next, they put the program's CD into the CD/DVD drives and listened to the robot guide showing them how to take the test. They then took the test within a span of 60 minutes. The pre-test did not provide subjects with annotations; so, they had to draw on their prior knowledge when approaching the test. At the end of the session, the subjects' profiles were analyzed and their mean scores were obtained. The result of pre-testing revealed that the groups delivered a poor performance on the test (X 1 = 12.17, X 2 = 12.47, X 3 = 13.43). To determine the groups' homogeneity, the Levene's Test of equality of variances and an ANOVA analysis were then used. The results of the analyses showed that the three groups were homogeneous at the beginning of the experiment (p > 0.05).
The three groups then received treatment on the same passages under one of three conditions. The treatment session began right after a one-hour break. The subjects should not have left the lab, as they might have remembered the keywords and looked them up in the dictionary. The first experimental group received treatment on the passages through a multimedia program drawing on pcito-textual glosses where a combination of textual definitions and relevant pictures would pop up on the screen by moving the mouse cursor over the keywords. Likewise, the second experimental group received treatment on the same passages through a multimedia environment using pictorial annotations where only pertinent images of the concepts associated with the very keywords would pop up. The third group, in a similar vein, received treatment on the same passages through textual definitions (see the appendix).
Both the question and passage templates appeared simultaneously on the screen so that subjects feel as if they were taking a real exam. The built-in countdown timer controlled the amount of time spent on the test across the three groups under study. When the limit reached, the programs would automatically save the students' answers in a log file and then move on to the next passage. There was no "Back" button; however, a warning message would pop up requiring the subjects to confirm whether or not they wanted to move on. Like the pre-test session, the treatment-post-test session lasted for 60 minutes where the subjects spent seven minutes reading the passages and five minutes answering the questions. At the end of the experiment, the subjects' profiles were analyzed and their mean scores were obtained.

Results
Tables 1 and 2 below show the results of the pre-and the post-tests respectively: Insert Table 1, Table 2 Here In the tables, "1" represents the experimental group who received treatment through picto-textual glosses, "2" refers to the experimental group who used pictorial glosses, and "3" denotes the one who used textual glosses. As shown in table 2, all the three groups obtained higher mean scores on the post-test as compared with the pre-test means. This implies that the glosses were effective enough to improve the subjects' comprehension. Moreover, as the table shows, the difference among the means was statistically significant (p < 0.05). A glimpse at the Scheffé Test results and the homogeneous subsets table (Table 3 below) further reveals that all the three means were significantly different from each other. No confidence interval contains zero, and the probability value for each set of comparison is smaller than our alpha value. This implies that the three types of glosses differed in their effectiveness in adding in subjects' comprehension.
Insert Table 3 Here This research sought to find an empirically justified answer to the following question: Q: Is there any significant difference in the use of the multimedia program drawing on textual glosses, the one using pictorial glosses, and the one using a combination of textual and pictorial glosses in aiding in EFL learners' reading comprehension?
The answer is "yes", as the three types of treatments were shown to differ in their effectiveness in aiding in subjects' comprehension. Accordingly, the null hypothesis formulated a priori, which assumed no significant difference in the use of the three types of multimedia glosses, was rejected. The experiment showed that the experimental groups who received treatment through pictorial glosses and a combination of textual and pictorial annotations outperformed the one who received treatment through textual glosses on the post-test. One rough explanation is that visuals are more readily remembered than are words. They might persist in the working memory, and hence information is more readily transferred to the long-term memory. Consequently, learners might arrive at a deeper understanding of the passages as they keep the key meanings in memory without losing their train of thoughts. Underwood (1989) argues that "A commonplace principle of human learning is visual memory. We remember images better than words. Hence, we remember words better if they are strongly associated with images" (p. 19). Some studies (Paivio, Smythe & Yuille, 1968;Denis, 1982, as cited in Chun & Plass, 1997Neu & Stewig, 1991, as cited in Bazeli & Olle, 1995Iheanacho, 1997;Johnson-Glenberg, 2000;Al-Seghayer, 2001;Babaie, 2010) corroborated this view.
Another justification is that visuals might help learners better decipher the meanings of keywords in reading passages. According to Babaie (2010), visuals, in antithesis to textuals, are more elaborate in nature, and hence might better reveal the underlying concepts with which words are associated. When applied to reading comprehension, students might, then, arrive at an even better understanding of reading passages, as the meanings of keywords are more effectively revealed to them. More studies are required to corroborate this view.
Yet another explanation is that pictures are effective attention-getting devices. Day (1982) argues that visuals have the potential to arouse students' curiosity as they attempt to analyze the concepts with which pictures are associated. This stimulation of curiosity might, then, more effectively focus learners' attention on the subject matter being introduced, which in the long run might lead to an even more effective acquisition of information. When applied to reading comprehension, the researchers of the present study contend that increased levels of attention on keywords can persist in learners' working memory, thus making the associated meanings more memorable. Accordingly, as students more readily keep track of the meanings in their memory, they may arrive at an even deeper undertraining of the passages.
The present study further showed that the experimental group who received treatment through picto-textual glosses outperformed the other two groups on the comprehension test. One explanation, as the researchers argue, is that relative to the pictorial-gloss condition, a combination of both textual and pictorial definitions might even better reveal the underlying meanings to learners. This added elaboration might, then, lead to an even deeper understanding of the reading passage.
Another justification is that the associations that are made between the textual and visual representations of the keywords in the working memory make the associated meanings even more memorable. When students quickly access the meanings in their memory, they might arrive at an even deeper understanding, as they can now more concentrate on the main ideas rather than the meanings themselves. The idea underpins Paivo's (1971Paivo's ( , 1986Paivo's ( , 1991 Dual Coding Theory (DCT) contending that pictures and words activate different visual codes known as "imagens" and verbal codes called "logogens" in the visual and verbal memories respectively. It is further hypothesized that three types of processes occur between these two memory modules and within each system: The first type is the "representational processing" that takes place between the incoming stimuli and the verbal or visual memory. For instance, seeing or hearing the term "heron" will activate the corresponding verbal code (heron) in the verbal system, while seeing the picture of this bird will readily activate the pertinent imagen in the visual memory.
The second type, the "referential processing", occurs between the verbal and visual memories. For instance, seeing or hearing the term "heron" will activate the relevant logogen in the verbal memory, and this will in turn activate the corresponding imagen in the visual memory. The reverse is also true. Seeing the picture of a heron will ultimately activate the logogen of the bird in the verbal memory.
Finally, the third type, the "associative processing", refers to the processing of information within each memory module. For instance, within verbal memory, a logogen like "accessory" might be associated with a number of other logogens like "a supplement", "an adornment", "an accomplice", etc. in different contexts. A previously activated logogen might, then, help activate other logogens associated with it (Sadoski & Paivio, 2004).
Dual Coding Theory thus suggests that pictures are more readily retrieved from memory than are words (Paivio, 1986), and that recall is enhanced thanks to word-image associations (Unnava & Burnkrant, 1991;Schmitt, Tavassoli, & Millard, 1993). When applied to reading comprehension, it can be contended that the combination of textual and visual pieces of information makes keywords more elaborate and thus more memorable. The meanings of the keywords are, then, more readily retrievable as learners go through the reading passage. This ease of access will lead to a deeper understanding in the long run, as students more effectively concentrate on the main ideas without losing their train of thoughts.
Still another justification is that when two memory modules are addressed instead of one, cognitive overload is less likely to happen (Baddeley, 1997;Sweller, Van Merriënboer, & Paas, 1998). Consequently, the limited capacity of the working memory can be allotted to higher-order comprehension skills. Convictions are strong that the more capacity is allocated to comprehension processes, the deeper the understanding of the passage. Due to a paucity of research, however, further studies are required to substantiate such claims.

Conclusion and Pedagogical Implications
This research favored the use of the picto-textual gloss as the most effective type of vocabulary annotation aiding in reading comprehension. Teachers or teachers as designers can author customizable pieces of multimedia courseware where passages on a variety of topics are annotated through picto-textual glosses to aid in both vocabulary acquisition and reading comprehension. This experiment thus suggests that the traditional guessing technique be supplanted by multimedia glosses of this type, as learners can more effectively decipher the meanings of keywords, more readily retrieve them from memory, thus freeing up the working memory's capacity for processing higher-order comprehension skills. Increased levels of attention together with ease of access to words' meanings might in turn help learners arrive at an even deeper understanding of the passages.

Suggestions for Further Research
The present research focused on the effects of multimedia glosses on the reading comprehension of intermediate-level EFL learners. It might be intriguing to explore the effects of glosses on the comprehension of students of other levels of language proficiency. This experiment was a one-shot study. Follow-up research should be longitudinal, thus exploring the likely contributions of glosses to reading comprehension over a relatively long span of time. Only through such experiments can one accurately estimate the efficiency (or inefficiency) of multimedia glosses. Moreover, there are other types of glosses, such as aural and video glosses, whose effects on learning can be investigated. According to Al-Seghayer (2001), static pictures and streaming video differ in some respects. Some studies favored the use of video in language pedagogy. Accordingly, one may replicate the present study, exploring the effects of a variety of glosses on different facets of language learning. Furthermore, the subjects of this study comprised mixed groups of male and female participants. Gender-oriented studies on reading comprehension have yielded contradictory results; some have favored males and others have favored females (Bügel & Buunk, 1996;Myers, 2002;Brantmeier, 2003;Pae, 2004). No research has yet explored the likely effects of gender differences on variations in subjects' performance on learning through glosses in multimedia environments. It is likely that some types of glosses prove more beneficial to either of the sexes. Future studies should address the issue.

Appendix: The Multimedia Courseware
The Login page The guide described how learners could use the glosses and approach the test.
In Picto-Textual-Gloss Multimedia Courseware, both the pictorial and textual definitions were presented to the subjects.