A Meta-Analysis : Improvement of Students ’ Algebraic Reasoning through Metacognitive Training

Although algebraic reasoning has been considered as an important factor influencing students’ mathematical performance, many students struggle to build concrete algebraic reasoning. Metacognitive training has been regarded as one effective method to develop students’ algebraic reasoning; however, there are no published meta-analyses that include an examination of the effects of metacognitive training on students’ algebraic reasoning. Therefore, the purpose of this meta-analysis was to examine the impact of metacognitive training on students’ algebraic reasoning. Eighteen studies with 22 effect sizes were selected for inclusion in the present meta-analysis. In the process of the analysis, one study was determined as an outlier; therefore, another meta-analysis was reconstructed without the outlier to calculate more robust results. The findings indicated that the overall effect size without an outlier equaled d=0.973 with SE=0.196. Q=20.201 (p<.05) and I=0.997, which indicated heterogeneity of the studies. These results showed that the metacognitive training had a statistically significant positive impact on students’ algebraic reasoning.


Introduction
In recent decades, the development and proficient use of algebraic reasoning has been considered a critical component for success in mathematics because it equips individuals with the ability to transition from learning and employing basic arithmetic to understanding and making use of more complex mathematical concepts (Kaput, 1998).Mathematics educators and researchers have advocated the inclusion of algebra in early childhood mathematics curricula to foster students' basic algebraic reasoning skills, which must be continually developed as students are exposed to increasingly complex mathematics in later grades (National Council of Teachers of Mathematics [NCTM], 2000).Despite the recent effort to develop and improve students' algebraic reasoning, thereby allowing for a deeper and more complex mathematical conceptualization (Blanton & Kaput, 2005), a number of United States (U.S.) students have not demonstrated proficient algebraic reasoning (Silver & Kenny, 2001).International assessments such as Trends in International Mathematics and Science Study (TIMSS) have shown that U.S. students demonstrate low achievement in algebra, particularly in algebraic reasoning (Mullis, Martin, Foy, & Arora, 2012).Researchers (e.g., Kuchemann, 1981;Macgregor & Stacey, 1997) also pointed out that a large number of students struggled to comprehend algebraic concepts that required algebraic reasoning.Many students encounter difficulties when transitioning from arithmetic reasoning to algebraic reasoning (Organization for Economic Co-operation and Development, 2003).Therefore, the effective guidance for development of algebraic reasoning should be implemented in practical teaching and learning situations.

Algebraic Reasoning and Metacognitive Training
Algebraic reasoning is an extended mathematical concept of arithmetic reasoning.When children solve problems such as 5+ ( ) = 12, they use their arithmetic reasoning to count and to conduct mathematical operations.Students can transition easily from arithmetic reasoning to algebraic reasoning if they deal with simple unknown quantities with numbers.Algebraic reasoning is more dynamic compared to arithmetic reasoning, which describes static situations (Kramarski, 2004).In particular, algebraic reasoning incorporates (1) generalized arithmetic, (2) generalized patterns (functional relationship), (3) models for expressing generalization, and (4) mathematical systems (Kaput, 1998).Students who develop concrete algebraic reasoning can generalize specific situations to mathematical ideas in increasingly formal ways from a specific situation and express these generalized ideas through discourse with others (Kaput, 1999).Algebraic reasoning, which was generalized from the particular situations, can help students "to understand patterns, relations, and functions; represent and analyze mathematical situations and structures using algebraic symbols; use mathematical models to represent and understand quantitative relationships; and analyze change in various contexts" (Friel et al., 2001, p. 2).Although students may have the ability to algebraically reason, their level of algebraic reasoning may be insufficient to support their understanding and use of mathematical concepts included in their grade level.Findings from previous studies indicated that many students did not engage in age-appropriate algebraic reasoning and struggled when the mathematic problems required them to transition from arithmetic to algebraic reasoning (Silver & Kenny, 2001;Warren, Cooper, & Lamb, 2006).Expanding their reasoning from static situations to dynamic situations, from concrete objects to formal symbols, and from specific to generalized thinking was a difficult process for many students (Sfard & Linchevski, 1994).Like this, transferring students' prior knowledge to new situations had a limitation such as inability to identify the similarities and differences between elements in situations (Salomon & Perkins, 1987).These difficulties impaired students' algebraic learning (Kramarski, 2004) and negatively affected their academic achievement in algebra (Warren, 2003).Therefore, developing and improving students' algebraic reasoning is critical to ensure their future academic success in mathematics, and the implementation of effective teaching pedagogies in mathematics classrooms is necessary for students to build concrete algebraic reasoning.
Recently, metacognition has received much interest and attention for its role in improving students' algebraic reasoning in mathematics classrooms (Kramarski & Ritkof, 2002).Metacognition includes students' thought processes and beliefs that enable them to regulate their learning activities (Schoenfeld, 1987).Some researchers (e.g., Kramarski & Mevarch, 2003;Salomon & Perkins, 1987) have made a distinction between general metacognition and domain-specific metacognition.General metacognition refers to being able to regulate problem-solving processes regardless of the specific situation.Domain-specific metacognition focuses on the unique characteristics of each situation among diverse/complicated situations (Kramarski & Mevarch, 2003).During the metacognitive processes, both knowledge and cognitive skills are planned, monitored, analyzed, evaluated, and reflected by students based on their own goals (Schraw, Crippen, & Hartley, 2006;Zimmerman, 2008).Using this metacognitive process positively impacted students' academic achievement (Helms-Lorenz & Jacobse, 2008).In addition, these processes encouraged students' willingness to learn what was related to improving self-efficacy (Kramarski, 2004).These positive results of students' use of metacognition processes stimulated further interest in educational research related to the use of metacognitive training, especially implementation of metacognition through actual practice (Kramarski, Mevarech, & Arami, 2002;Schoenfeld, 1987).Metacognitive training is one practical teaching method in which the concept of metacognition is adopted.
Metacognitive training provides small group activities for students to ask and answer questions by themselves.These self-addressed metacognitive questions primarly consist of questions that naturally arise and that relate to the connection between prior and new knowledge and the use of appropriate problem-solving strategies (Kramarski & Mevarech, 2003).Metacognitive training strategies encourage students to "make them exert the initial extra effort, and prolonged training to guarantee the smooth and maintained application of metacognitive activity" (Veenman, Van Hout-Wolters, & Afflerbach, 2006, p. 9).Through use of these strategies, students could develop advanced in-depth learning, active cognitive processing, and better mathematical understanding, thus improving algebraic reasoning (Zimmerman, 2002).For these reasons, metacognitive training has been considered an effective tool for improving students' algebraic reasoning.Therefore, it is critical to investigate the provision of metacognitive training to improve students' algebraic reasoning.

Purpose of the Present Study
The purpose of the present study was to examine the impact of metacognitive training on students' algebraic reasoning and to report quantitative data in sufficient detail for the calculation of effect sizes by applying a meta-analysis.The following research question was used to guide our study: Does metacognitive training positively affect students' algebraic reasoning?

Inclusion Criteria and Literature Research
A meta-analysis was conducted to determine the impact of metacognitive training on students' algebraic reasoning and compile crucial insight related to this topic from what might otherwise be a disparate literature (Bangert-Drowns, Hurley, & Barbara, 2004).The electronic databases of ERIC, JSTOR, and Google scholar were used to compile data for the present study.The researchers searched for relevant studies using the following keywords: "algebraic reasoning", "algebraic thinking", and "algebraic achievement" with "metacognitive training", "metacognitive guidance", "metacognitive instruction", and "metacognition".A backwards search was also applied to collect additional data using the studies cited in other articles.The researchers found 121 abstracts based on this process.The abstract and conclusion of each study was reviewed, and the studies were accepted based on the following inclusion criteria: (1) published in English; (2) involved students in grades K-12; (3) measured examinations of students' algebraic reasoning; (4) included the use of metacognitive training; and (5) and published in peer-reviewed journals.Of the 121 abstracts that were found, 78 abstracts, and therefore studies, were excluded based on this inclusion criteria.In addition, of the 43 remaining studies, 25 studies were excluded because of their lack of information for meta-analysis.For instance, some studies did not demonstrate effect sizes or provide enough statistical information to calculate effect sizes.Therefore, 18 studies with 22 effect sizes were included in the present study.

Coding Procedure
One member of the research team in the present study conducted the initial coding of the studies, after which the other three researchers separately evaluated any points of disagreement.The inter-rater agreement related to determining the continuous variables was .90.If there were more than one effect size within a single study because of different groups, we considered each effect size as an individual study.Note.K=Kindergarten, E=Elementary, M=Middle, H=High.

Computing Effect Sizes
Eighteen studies with 22 effects sizes were included for meta-analysis in the present study.Kramarski et al. (2002), Kramarski and Mevarech (2003), Pennequin et al. (2010), andVula et al. (2017) contained more than one effect size for different participants within a single study, including for example, effect sizes for students in different grades.In this case, we reported each effect size as an individual study.Note.Index '1' is an overall effect size and CIs of the studies.

Statistical Analysis
The overall effect size was calculated and presented using Excel.Each study was coded independently by the researchers using coding sheets to document the following information: author, sample size, grade, intervention, instrument, mean (M), standard deviation (SD), Cohen's d, SE, and 95% CI.The effect sizes, Cohen's ds, and 95% CIs, were calculated by using Ms, SDs, and sample sizes when the studies did not have effect sizes.The effect sizes were weighted by sample sizes to avoid sampling bias (Lipsey & Wilson, 2001).Then, the overall mean effect sizes and SEs were calculated using the weighted effect sizes.Forest plots were used to illustrate the relative strength of the intervention (metacognitive training) to the dependent variable (students' algebraic reasoning).The SE of overall effect size was calculated and used to demonstrate how the mean of each effect size located when examined as a comprehensive unit of effect sizes rather than analyzed as separate effect sizes between distributed studies.In addition, Q-test and I 2 were computed to test the heterogeneity across the effect sizes.If the p-value of the Q-test was lower than 0.05, we considered the studies statistically significantly similar.In addition, I 2 described the percentage of total variation across the studies.

Results
The homogeneity test for the 22 effect sizes was significant, (I 2 <0, Q=13.886, p<.05), indicating between-studies heterogeneity.Negative I 2 considered as zero (Higgins, Thompson, Deeks, & Altman, 2003).Therefore, we could conclude that there was no observed heterogeneity across the studies.The overall effect size about the impact of metacognitive training on students' algebraic reasoning across 18 studies (k=18, n=1421) was positive, d=2.284 with a SE=1.298. Figure 1 showed a forest plot of the included studies.demonstrated that the positive impact of metacognitive training on students' algebraic reasoning was statistically significant: the overall effect size was d=2.284 with SE=1.298.In particular, the overall effect size without an outlier was d=0.973 with SE=0.196.This result also showed the statistically significant positive impact of metacognitive training on students' algebraic reasoning.In fact, the overall effect size of the reconstructed meta-analysis was even more robust than the overall effect size of the initial meta-analysis.The Q-tests also made clear that assuming a random-effects model for these analyses is appropriate because the variation in the effect sizes cannot be explained solely by sampling error.The I was close to zero, which also strongly supported the overall effect size.
From the findings, the importance of metacognition for algebraic reasoning might be explained.Metacognitive training is a practical training method in which the concept of metacognition is adopted.There were many different types of metacognitive training examined in the included studies for this meta-analysis, such as computer-based (online) training, individual training, collaborative training, and self-regulation.Regardless of the method types, metacognitive training helps students develop their algebraic reasoning.Being exposed to metacognitive strategies could be the reason for improvement in algebraic reasoning.However, the degree of improvement in one's algebraic reasoning might vary depending on the type of metacognitive training methods employed.For further study, moderator analysis including the types of metacognitive training can be useful to determine which type of metacognitive training most positively impacts students' algebraic reasoning.
There was no common instrument used among these studies to measure students' algebraic reasoning.Although many researchers pointed out the importance of algebraic reasoning, some of them used instruments created for assessing students' mathematics achievement or algebraic achievement.A number of researchers seemed to prefer using qualitative research methods to explore the effect of metacognitive training, and many studies were excluded from the meta-analysis in the present study because they did not provide quantitative results that could be used to obtain effect sizes (e.g., Berger & Karabenick, 2016).As a result, the present study includes only 18 studies with 22 effect sizes.This limitation restricts the overall examination of the improvement of students' algebraic reasoning through metacognitive training to which the results of this study can be generalized.

Table 1 .
Included primary studies

Table 2 .
Table 2 demonstrates each study's Cohen's d, standard error (SE), and 95% confidence interval (CI).Cohen's d, SEs and CIs of included studies