Enhancing Students’ Accuracy in Tests and Understanding of the Main Ideas of Reading Materials by Retrieval Cues in the Textbook

Since understanding reading assignments is important to succeeding in school, improving the way that text is arranged in books would be an efficient way to help students better understand the material and perform well on tests. In this study, we asked students to read two original and two rearranged historical passages, in which rephrased questions from the test of the passage were embedded in the reading. Those restated questions were embedded into the corresponding sections in the text where the answers for the questions were located. After the reading, they were tested on their understanding of the material with multiple choice questions. All participants took an identical test. According to the results, both groups’ accuracy was higher for the questions of the two rearranged passages in each group. In other words, the textbook makers should consider putting retrieval cues for the test into the textbook in order to make the textbooks more readable. Thus, the students’ retrieval of the concept and their performance on the test will be enhanced.


Introduction
Rather than just testing, introducing retrieval cues into tests can promote retention in useful and practical applications, especially in school settings (Roediger & Karpicke, 2006a). To clarify, retrieval practice is a study strategy, which bring information to the forefront of the thoughts to enhance and boost the effects of learning, improving the process of recalling information and forcing the person studying to extract knowledge to examine the knowledge (Abbott, 1909). The influences of integrating retrieval practice and education have been researched for a long time and proven largely beneficial for learning processes (Nunes & Karpicke, 2015). The impact of retrieval practice on retention is strong, and thus cognitive psychologists recommend that retrieval induced testing could be used as a way to improve learning in the classroom, especially in classes that involved more with reading, such as humanities classes (McDaniel, Roediger, & McDermott, 2007;Roediger, Putnam, & Smith, 2011).
Knowing the effect of retrieval practice in tests is helpful for students to perform better in assessments. However, little research has been conducted on the ways that embedding questions in texts can affect retrieval processes. The arrangement of the text from the reading material and the questions from the test can be related to each other to trigger the readers' (in this case, the students') retrieval mindset. In other words, the text and the test questions can serve as the retrieval cues for one another. This practice of embedding more questions into the text, which has the potential to improve textbooks, would be relatively easy to achieve. By embedding the rephrased test questions into the text, the passages that included the retrieval practice can be used to teach the readers the major concepts from the passages. As a result of adding retrieval questions in the textbooks, the students might possibly have a better performance in the future standard tests, especially in the multiple-choice questions.

Method
The experiment examined whether embedding questions in the test would produce good retrieval practice effects on students by examining the test results of the student subjects. Subjects were randomly divided into two groups, and each finished the reading section of the experiment first. This reading section consisted of four passages. The passages were taken from different time periods from one high school world history textbook, and on average, the length of the passage is 500 words. The text of two passages did not change, while rephrased questions from the test, as retrieval cues, were added to the appropriate locations in the other two passages. In other words, the rephrased questions were added into the places where the answers of the questions were located. The test questions were provided by the history textbook for the four passages.
After reading the four passages in 12 minutes (3 minutes per passage), the subjects then took a 32-question test (8 questions for each passage, at the same difficulty level) in 15 minutes to evaluate their understanding of the passages (two groups took the same test). The time limit of the reading and testing sections were designed with the average speed of reading of the high school student subjects, according to the background research (in which the subjects were asked to read a separate 800-words reading passage and recorded their time spent reading the passage, and their average reading speed was calculated) in this experiment about student subjects' average reading speed. The first group read the original passages 1 and 4 and the rephrased passages 2 and 3, while the second group read the original passages 2 and 3 and the rephrased passages 1 and 4. Dividing the subjects this random way with rephrased passages from different time periods could help to test the accuracy of the result. This strategy was useful, because this can eliminate the factor that students might do better on certain time periods (the possibility of having previous knowledge about the time period). Thus, by assigning students randomly into two different groups with passages from different time periods, a bias on the result will be limited. During the test, subjects were cued by the questions to recall the important information mentioned in the two rephrased passages of each group, since they remembered the rephrased questions (retrieval cues) from the passages. Thus, when the students were doing the test, they see the retrieval cues again, which in turn enabled them to remember the right corresponding answers that cued them from the reading.

Subjects
50 subjects (25 female and 25 male, ages 16-18, mean age of 17) were recruited, taking the same English test for this research. The subjects from China spoke English as their second language, while those from the United States were native speakers. However, the two groups of subjects have similar English level, since all the Chinese subjects come from high schools in the United States, whose average humanities grade was 3.85 and lived and studied in America for at least 2 years. The American students also had an average humanities grade of 3.85, and thus, the English ability of the Chinese international students and the American native students was comparable for the results of the experiment.

Design
The research was an experimental type. One variable of the experiment is the passage type: original passage or passage with the embedded questions. The passage type is a within-subjects independent variable. The other between-subjects variable is the order of passages presented for the subjects in the reading section with two groups, 1 and 4 rephrased or 2 and 3 rephrased. Therefore, the independent variable is the type of the passage, original or rephrased, while the dependent variable is the final accuracy of the test.

Materials
Materials of the experiment consisted of four passages from a high school level world history textbook. These historical passages were selected because this type of writing is straight forward compared to other types such as literature passages, since historical passages consists of more objective illustration and introduction of the historical events. Historical passages also involve cause and effect, which could help to demonstrate the subjects' complex understanding of the reading material. In each group, two passages were original, while the other two passages had the embedded questions. Thus, there are overall four original passages and four rephrased passages in total (but each time, only 2 passages of each type were given to each group of subjects). The historical passages were written about different time periods. The first passage was about the history of ancient Babylon, which described the founding of the Babylonian society from Old Babylon to the later Babylon, and the political development of the ancient society. The second passage is about feudalism and aristocratic class in Medieval Europe. The third passage discusses the Spanish American War and the Treaty with Spain in 1899. The fourth and final passage illustrates the Church-State relationship in Europe before the 15th century. Both the original passages and the passages with questions have an average length of 500 words. Since the four passages have distinctive themes, the accuracy of the results could be fostered. The multiple-choice questions asked simple main ideas from the passages for a retrieval test and some comprehension questions to check subjects' understanding and their retrieval of the details from the passages. The experiment was conducted in Google Form. To monitor the experiment in order to obtain accurate results, the subjects were asked to video record themselves and their screens during the reading section and the test section. In other words, with the requirement of recording, the subjects would finish each section within the time limit. In the end, by examining their screen records and the video records of the subjects, the time used for the reading section and the test section could be monitored and evaluated. Therefore, according to the examination of the video recordings, cheating such as to go jel.ccsenet.org Journal of Education and Learning Vol. 8, No. 3;2019 back to the reading section or to search about the correct answers online were prevented (no subject cheated).

Procedure
The entire experiment was conducted within three months during the summer of 2018, which enabled the researcher to collect enough sample data. At the beginning of the experiment, subjects read the four passages within 12 minutes. The sequence of the passages was the same for each group, but in each group, the two rephrased passages were different. Subjects were instructed to memorize and understand the main ideas of the passages in order to retrieve the memory while they were taking the multiple choice test. In the test, there were 8 questions for each passage. The subjects were notified by an announcement to avoid as much guessing as possible and keep time in mind. The questions served as cues for the subjects to recall the corresponding passages, since in the text, rephrased questions from the test were located near by the answers. Then, after 12 minutes, the subjects finished their answers and submitted their results. Every part of the experiment was meticulously monitored for the accuracy of the results.

Results
All results were significant, at the .05 level. The bar charts in Figures 1 and 2 and Table 1 show the results of the experiment and the comparison between each group's performance (their accuracy on the test).
On the test, subjects responded to two sets of questions for the two original passages and two sets of questions for the two rephrased passages. By analyzing results from these responses, we can determine that the number of errors for the rephrased question were, overall, smaller than the number for the original passage questions (Figures 1 and 2). From the calculations from the result of  Table 2 suggests, the average errors in all the passages have some variation, Table 3 still suggests that the average error of the rephrased passages (2 and 3 for Group 1, 1 and 4 for Group 2) is lower than that of the original passages.  Moreover, the calculated p-value, 0.000014, indicates a real positive effect of the retrieval devices on the student's understanding and performance on the multiple-choice test. In other words, the small p-value (≤ 0.05) indicates strong evidence against the null hypothesis. Practicing retrieval enhanced the general accuracy of the test questions in both groups. Since the test is a multiple-choice based assessment, the calculated result is objective with the analysis of only the accuracy of the multiple-choice answers. Therefore, the results were proved to have enough accuracy to draw a conclusion.

Discussion
Results from the experiment indicate that retrieval cues in questions induced a retrieval response, increasing students' performance, accuracy, of the test. The retrieval cues from the two rephrased passages in each group and the questions in the test corresponded with one another, enabling students to relate and recall information they had read several minutes before in the previous the passages embedded with section. The students had the information they learned in mind, and with the help of retrieval cues, they produced a response more accurately in test section for the rephrased passages. Consistent with previously published work, the results clearly show that establishing and creating overt retrieval traits in the text, which students used to study for the material and for the assessment tests, could enable students to perform more accurately on tests as a result of corresponding retrieval cues (Smith, Roediger, & Karpicke, 2013). Future studies should consider extending the time between the reading and test section to check the retrieval devices' effect on long-term retention. Checking the effect and benefits of retrieval cues on other field of study other than heavily text-based humanities field can help to illustrate the real advantage of retrieval cues. This could help textbook authors and editors develop writing strategies so as to best help students retain knowledge. Moreover, students could use retrieval cues for fostering their understanding of the text on their own, since retrieval practice in the text had proven advantages in increasing accuracy in tests in this experiment. While reading the text, to think about the possible specific questions that the test makers would ask in the passages and to answer the questions on their own could possibly improve their performance as well. In this way, using a similar method like that of the experiment could contribute the successful retrieval in the same way.

Study Findings
This study proved again the retrieval practice's positive effect on retention and assessment accuracy. This research can bring more attention to the arrangement of textbook passages for textbook makers in the future to write better passages in order to increase the students' accuracy on tests. However, placement of questions in chapters is different from the placement of retrieval questions inside of the individual passages. Placing questions directly in the text can reinforce retrieval of specific concepts, although placing questions by the end of the chapters can contribute to the learning process and memory retrieval (Uner & Roediger, 2018). According to the result of the experiment, the placement of questions in individual passages increases students' accuracy on more specific concepts from the passage rather than the general cause and effect understanding promoted by the chapter questions. Thus, the study revealed the retrieval effect of the placement of specific questions related to the content of historical passages in the textbook, which helped to increase student's accuracy of the assessment of the material.

Limitations
The limitation of this research is that the retrieval questions in the text and the multiple-choice questions in the test are very similar to each other, which might lead to an easy test for students, since the questions embedded in the text are the paraphrased version of the other questions from the test. This might enable the students to quickly find the accurate choice in the test through reading and memorizing the answers from the test. Thus, the test might need some changes in order to illustrate the student's real retrieval after the aid of the questions in the texts rather than using a short-term memorization of the paraphrased questions and answers. Changes in question format or creating more difficult question on subtle details of the passage in future research can help to contribute to a more complex understanding of the reading material instead of just the memorization of the text. Moreover, this research did not reflect that students had proven their complex reading comprehension. This is the limitation of multiple-choice-based tests that students' test scores cannot reflect the complexity of their view about the passage or the demonstration of their own understanding of the reading. In fields like humanities, not only remembering the events in history is important, understanding the history and the connections between events is also important. Therefore, although the study reflected the advantage of adding retrieval cues into text could enhance student's performance and accuracy on tests, especially multiple-choice tests, the limitation of retrieval cues have to be addressed in the future studies in order to improve student's understanding of the text and the complexity of their own thoughts about the material.