The Effects of Problem Posing on Student Mathematical Learning: A Meta-Analysis

The purpose of the study was to meta-synthesize research findings on the effectiveness of problem posing and to investigate the factors that might affect the incorporation of problem posing in the teaching and learning of mathematics. The eligibility criteria for inclusion of literature in the meta-analysis was: published between 1989 and 2011, reported problem posing as an intervention, employed experimental research design, and provided data necessary to compute effect sizes. The large positive effect sizes (Hedges’ g) showed that problem posing activities provide considerable benefits for: mathematics achievement, problem solving skills, levels of problems posed, and attitudes toward mathematics. Several noteworthy limitations of this study were discussed.


Introduction
In recent years, mathematical problem posing has been gaining considerable attention as a useful cognitive activity along with problem solving. The National Council of Teachers of Mathematics (NCTM) highlighted the importance of problem posing as a classroom intervention strategy for reforming school mathematics. Mathematics educators have acknowledged problem posing as a worthy intellectual activity based on constructivist theories of teaching and learning (Silver, 1994). According to English (1997), when students pose their own problems, they can enhance their mathematical knowledge, stimulate critical thinking, and improve computational skills by exploring their curiosity about specific mathematics concepts.
Problem posing is considered a developmental tool for critical thinking (English, 1997;Lowrie, 2002) because it can help students extend what they know in order to develop mathematical fluency and engage them in higher-order thinking (NCTM, 2000). In order to create an effective problem, posers must have imaginative skills that can be developed through the process of problem solving (Kilpatrick, 1987). Lavy and Bershadsky (2003) stated students not only needed to think mathematically but also think creatively when reformulating and generating a new mathematical problem. These researchers believed that students who were engaged in problem posing activities became enterprising, creative, and active learners.

Problem Posing in the Teaching and Learning of Mathematics
Research on problem posing has been shown to have positive outcomes on students' knowledge, problem solving skills, problem posing abilities, creativity, and disposition toward mathematics (e.g., Cai, 1998Cai, , 2003Cai & Hwang, 2002;English, 1997English, , 1998Lavy & Bershadsky, 2003;Moses, Bjork, & Goldenberg, 1990;Silver, Mamona-Downs, Leung, & Kenney, 1996;Stoyanova, 1999;Yuan & Sriraman, 2011). For instance, Lavy and Bershadsky (2003) discovered that the use of problem posing activities contributed to the development of an individual's mathematical knowledge. In this study, the What if not? strategy (Brown & Walter, 2005) was adapted into two learning workshops on complex solid geometry. The results showed that students have deeper understanding of geometry and strengthen some interrelated mathematical concepts which supported Moses et al.'s (1990) ideas of restructuring and connecting knowledge based on prior ones with the use of problem posing activities.
teachers incorporated a problem-posing intervention, classrooms became more student-centered and students were more actively involved in creating and solving their own problems. On the other hand, problem posing gave students ownership of the problems they had generated or formulated (Cunningham, 2004;Grundmeier, 2003;Within, 2006). Cunningham stated that when students created new problems, they increased their sense of responsibility as they constructed their own knowledge while critiquing and refining problems with their classmates. By encouraging students to share and critique their ideas during mathematical activities, a teacher can develop the dispositions and inquiry skills students need to help them develop as effective problem posers (English, 1997;Gonzales, 1998;Silver, 1994).
Despite the significant contributions of this cognitive activity for improving student learning in mathematics (NCTM, 2000), very few classroom teachers have students regularly create their mathematical problems (Crespo & Sinclair, 2008;Silver, 1994). In fact, students of all ages, including new teachers, have limited experience posing their own mathematical problems (Crespo & Sinclair, 2008). For instance, Silver et al. (1996) found that 53 middle school teachers and 28 prospective secondary school teachers were able to generate at least one mathematical problem but many were "ill-posed or poorly stated problems" (p. 305). The results of this study suggested that even though the participants lacked educational experience, it was not a constraint for adapting problem posing in classrooms. However, Silver et al. (1996) strongly believed that when teachers do not have a certain level of proficiency to pose mathematical problems, it is difficult for them to involve and engage in this activity.
Because teachers have an essential role in the implementation of problem posing in classrooms, they must develop skills in problem posing and be able to create tasks, which include situations that are suitable for student engagement and participation (Gonzales, 1996;Silver et al., 1996). In fact, the NCTM's Principles and Standards for School Mathematics (2000) stated that teachers are responsible for creating a learning environment that promotes student discourse and provokes their critical thinking through problem posing activities. Many mathematics educators, therefore, have suggested the incorporation of problem posing during teacher education for creating more opportunities for teachers and preservice teachers to learn and experience the process that will eventually impact their future classroom pedagogical strategies. In fact, Stoyanova (2005) argued that problem posing was similar to other skills that can be developed and nurtured over time from structured to less structured activities. With enough exposure to problem posing activities, anyone should be able to pose their own problems that are mathematically rich, realistic, and challenging (Chen, Van Dooren, Chen, & Verschaffel, 2010).

The Need for a Meta-Analytic Review
A large body of literature has examined how problem-posing activities were integrated in mathematics classrooms, however, much remains to be understood about the types of classroom settings, type of presentation, and students that would most benefit from teaching and learning mathematics with problem posing. To date, there is only one review of the literature available providing an overview of published articles on problem posing. That review was conducted by Silver (1994), who summarized the nature, processes, and perspectives of problem posing activities. He discussed some insightful findings from research articles and provided recommendation for further investigation. Yet, Silver's review was written 20 years ago, hence, a systematic approach for reviewing the body of empirical work is needed. The present study employed a meta-analysis methodology for integrating research findings on the effects of problem posing interventional strategies on student learning outcomes. It is the first meta-analytic review on this topic designed to help teachers and researchers in making decisions regarding problem posing as an effective classroom interventional tool for the teaching and learning of mathematics.

Purpose of the Present Study
With a growing number of studies focusing on problem posing as a classroom intervention, it becomes important for us to examine the effectiveness of this approach for improving student mathematics learning. Therefore, the purpose of this study was to systematically synthesize empirical research on problem posing as an intervention in mathematics classrooms. The term intervention, as used in this study, describes any type of problem posing activity or program that might affect student learning of mathematics. In addition, we investigated the factors that might affect the incorporation of problem posing into the teaching and learning of mathematics. This meta-analytic study addresses the following questions: (a) Does problem posing lead to improvement in student learning of mathematics? (b) What factors (e.g., interventional features, methodological, student characteristics) moderate the effectiveness of problem posing in mathematics classrooms? www.ccsenet.org/ies International Education Studies Vol. 7, No. 13;

Inclusion Criteria and Literature Search
In the U.S., problem posing gained considerable attention in 1989 when NCTM documented the reform vision for school mathematics with an endorsement incorporating a cognitive activity, problem posing with problem solving as a means of classroom intervention. Since that year, the research on mathematical problem posing has grown. For purposes of this meta-analysis, we located significant works starting in 1989 until 2011 in order to systematically review effects of the problem posing intervention on student learning of mathematics.
The first step was to determine the meta-syntheses inclusion terms. The subject term keywords "problem posing", "problem writing", "problem formulation", "problem generation" (with the Boolean operator of an asterisk) and "mathematics" (e.g., problem posing* and mathematics) were selected and used in the following web-based educational databases: Educational Resources Information Center [(ERIC) via EBSCOhost], ProQuest Dissertations and Theses (ProQuest), and PsycINFO (ProQuest). We explored the databases of returned citations looking for works that focused on mathematical problem posing as a classroom intervention. Additional studies were located by examining the reference lists of the selected studies. Also, a manual search was conducted of a number of leading journals in education, particularly in mathematics education such as The multiple searches yielded an initial pool of 3,730 works, however, many of these studies were duplicates, or not applicable to the topic under investigation, or untraceable. We scanned the title of each work and sorted the potential papers. This resulted in 167 articles from journals, proceedings, ERIC documents, and dissertations related to mathematical problem posing. The next step was to examine each article in depth, using the eligibility criteria. The eligibility criteria for inclusion were: (a) reported problem posing as an intervention without the aid of a computer system/program, (b) employed experimental research design (i.e., treatment and control groups or pretest-posttest group), (c) published in the English language, and (d) reported statistical data (e.g., mean, standard deviation, frequency, proportion, t-test, correlation coefficient) for calculation of effect sizes. Many articles were discarded because they adopted non-experimental research designs (n=71; e.g., Cai, 1998;Yuan & Sriraman, 2011), presented only theoretical frameworks and literature reviews (n=23; Christou, Mousoulides, Pittalis, & Pitta-Pantazi, 2005;Contreras, 2007), and documented pedagogical approaches (n=43; e.g., Cai & Brook, 2006). We also excluded two studies that did not report the appropriate statistical data to compute effect sizes (i.e., Barlow & Cates, 2006) and 13 studies that utilized a computer system/program to enhance problem posing (e.g., Hirashima, Yokoyama, Okamoto, & Takeuchi, 2007). After screening the abstracts and careful examination of each remaining work, only 14 individual studies were selected because they satisfied the requirements for this meta-analysis.

Coding Procedure
The retained studies were coded using a 35-item coding sheet, which was developed iteratively based on the information provided in the studies. The researchers coded the articles according to important variables that illustrated the characteristics of the study. For example, the independent variables coded in the studies were authors, year of publication, type of publication, sample size, grade level, task administration, instruments, type of treatment, and statistical analyses used. Some particular research studies did not provide enough information for coding; hence, we contacted the authors to gather data about their important variables and research design. The dependent variables included the type of student mathematical learning outcome (i.e., knowledge-based, ability-based, skill-based, attitude and belief) resulting from the problem posing intervention. We categorized the learning outcome as knowledge-based when a study reported an effect of problem posing on student achievement in mathematics content. For ability-based measure, we coded an intervention that had an impact on students' levels and kinds of problem they posed. When the participants were able to perform some computation and apply problem-solving skills, we classified them as a skill-based measure. Finally, data collected about students' attitudes and beliefs about or toward mathematics, we coded as an attitude-beliefs outcome.  In this meta-analytic study, two researchers reviewed and coded two articles for assessing inter-rater reliability. The inter-rater coding was 80% and discussions were conducted to resolve disagreements related to coding procedure resulting in a 100% agreement. Then, one of these researchers coded the remaining articles. The description of the selected articles is illustrated in Table 1. After the coding process, we computed an effect size index for each learning outcome in a study. According to Lipsey and Wilson (2001), the effect size was an essential feature of meta-analysis in order to statistically combine and compare the studies that use different measurement procedures. The calculation was based on the standardization concept of different statistical results in each of the previous research studies. This standardized score (effect size) was interpretable and meaningful when making assessments across the variety of studies.

Computing Effect Sizes
The obtained effect sizes were based on seven published articles, five doctoral dissertations, and a master's thesis included in this meta-analysis. These studies employed an experimental research design for two independent groups or matched groups (pretest-posttest) and reported frequencies, proportions, and/or means for illustrating "two variable relationships" (Lipsey & Wilson, 2001, p. 41). In order to accommodate the variability of study designs and data types reported in each study, three effect size indices were included in this meta-analytic study as presented in Table 2. Borenstein, Hedges, Higgins, and Rothstein (2009) stated, "A single systematic review can include studies that used independent groups and also studies that used matched groups. From a statistical perspective the effect size has the same meaning regardless of the study design" (p. 30). A transformation for adjusting the indices used for different types of data is needed before combining these estimates in a meta-analytic study (Borenstein et al., 2009;Lipsey & Wilson, 2001).
According to Lipsey and Wilson (2001), the standardized mean difference is appropriate when contrasting experimental and control group means by adjusting the standard deviation of these groups to make them www.ccsenet.org/ies Vol. 7, No. 13; comparable even if the studies were using different measurement. In addition, the computation of standardized mean difference effect sizes from summary statistics (e.g., t-statistic, chi-square) is preferable to account for covariates. Then, the standardized mean gain is used when contrasting the matched group means that utilized the pooled standard deviation and correlation between the pre and posttest (Lipsey & Wilson, 2001). The odds ratio formula is an effect size metric for independent group research studies that uses dichotomous variables for which the data were presented in terms of frequencies and proportions (Lipsey & Wilson, 2001). Two studies that utilized a matched group research design (pre-posttest only) but reported only proportions and frequencies as their results were excluded. These were excluded because we found it impossible to extract the data into a 2 x 2 table for finding an effect size index, dependent risk difference or dependent proportions difference (Sanchez-Meca, personal communication, November 27, 2011). In addition, we were unable to find any citations that offered the possibility of converting and comparing dependent risk difference into Hedges' g or odds ratio indices.

Statistical Analysis
With careful examination of the variation of data in each article, a table was developed in Microsoft Office Excel (Ms-Excel) version 2007 to organize the data for ease of computation. The calculation of effect size was performed on an individual basis because some of the studies employed more than one outcome measure of problem posing effectiveness within a study. First, we calculated the Glass' Δ effect size and standard error for each learning outcome in a study. The weight function was used to weight the effect sizes by sample sizes (Lipsey & Wilson, 2001). If the sample sizes were equal, the mean effect was the average of the effects (Lipsey & Wilson, 2001). Next, we applied the Hedges' g adjusted estimate to each effect size index for correcting the sampling bias which gave more weight to studies with larger samples (Lipsey & Wilson, 2001). If the study examined a key outcome construct in several ways, an average of the overall effect sizes was computed. Then, we adapted Hasselblad and Hedges' (1995) ideas of converting among effect sizes because some indices were on different scales, continuous and dichotomous (Lipsey & Wilson, 2001). For ease of interpretation of results and discussion, we converted all odds ratio to Hedges' g indices based on √ where and E g are odds ratio and Hedges' g effect sizes, respectively. In the final stage of data analysis, the Hedges' g indices were grouped into similar research outcomes for testing the homogeneity analysis (Q statistics) of the effect size distribution. The test provides information about the variations of the true effect sizes that might have occurred because of sampling error or other aspects (Borenstein et al., 2009). In addition, the data were imported into Statistical Package for Social Science (SPSS) version 17.0 (2008) and funnel plots were constructed to display the effect size distribution visually in order to detect publication bias as a potential threat to validity. According to Allen, Chen, Willson, and Hughes, (2009) publication bias happens when publication favors studies with statistically significant results over studies with non-statistically significant results. The funnel plot was based on the mean effect for each study against the total sample size, and the confidence intervals (Allen et al., 2009). Asymmetrical plots might indicate the possible presence of publication or other forms of selection biases; however, other factors such as the design, participants' characteristics, type of treatment, and true heterogeneity may affect the shape of the funnel plot as well (Tang & Liu, 2000).

Moderator Variables
We hypothesized four variables as potential moderators that could influence the size of effect from the problem posing intervention (a) educational settings (grade 3-12 versus teacher preparation); (b) research designs (group versus pre-post contrasts); (c) publication sources published versus unpublished); (d) presentation types (structured versus non-specific problem posing program). The moderator analysis was performed in Ms-Excel by comparing the average effect sizes of each key outcome measure in different categories that formed the levels of the moderator. Specifically, in this moderator analysis, we applied a fixed-effect model where in a Q test based on analysis of variance (ANOVA) was utilized for comparing two or more groups (Borenstein et al., 2009). For example, when comparing two groups (Group A with B), we computed the weighted sum of squared deviations of studies in each group (Q A , Q B ) and the sum of Q A and Q B (Q within ). Then, we calculated the weighted sum of squared deviations of the subgroup means about the grand mean (Q between ) and the weighted sum of squared deviations of all effects about the grand mean, Q. We ran moderator analyses for all learning outcomes and based on the calculated quantities, we hoped to determine whether the effect of a problem posing intervention differed in the two groups and moderators were identified in every learning outcome.

Description of Selected Studies
A total of 14 unique datasets (from 13 studies) were included in the current meta-analysis. Table 1 provided study details. Five studies were published between 2009-2010, two from Turkey, and one each from the U.S., Kazakhstan, and Australia. The remainder were published between 1994 and 2008 from diverse countries such as Australia (n=2), the U.S. (n=3), and one each from Oman and China. These datasets contained approximately 1,316 participants including school students and preservice teachers. Most of them contained sample sizes of less than 100, and three contained 200 to 327 participants. These studies focused on almost all levels of grade 3-12 education with 1,038 participants. The remaining studies were comprised of elementary and middle grade preservice teachers (n=278). Most of the participants did not receive any incentive in terms of monetary or course credits but for students in grades 3-12, parental consent was noted. For preservice teacher education programs, the problem posing activities were conducted in their mathematics methods courses, while in grades 3-12, the researchers utilized problem posing as an intervention strategy with arithmetic, algebra, number sense, and problem solving content. Many of the researchers incorporated problem posing using Brown and Walter's (2005) What if not? into their classrooms. Some adapted the free, semi-structured, and structured problem posing situations strategies by Stoyanova and Ellerton (1996) and others used non-specific problem posing interventions in their studies.
In terms of research design, the selected studies in this meta-analysis utilized quasi-experimental groups (n=11) and pre-post contrasts (n=2). Many of these researchers developed new instruments (n=6) for data gathering while others (n=7) used standardized (e.g., Iowa Test of Basic Skills-Problem Solving Subtest) or adapted instruments from previous research. In terms of research focus, most of the studies reported the effect of problem posing interventions on ability-based and attitude-beliefs learning outcomes. These included six studies that explored students' levels and abilities to pose problems as well their interest toward mathematics. The remaining studies focused on the effect of problem posing interventions on students' problem solving performance and mathematics achievement. Table 3 provides a general description illustrating the selected studies with comparisons between the weighted mean of effect sizes of the problem posing intervention based on student learning outcomes: knowledge-based (e.g., mathematics achievement), skill-based (e.g., problem solving performance), ability-based (e.g., levels of problem posed), and attitude-beliefs (e.g., interest and belief about/toward mathematics). The results showed that the problem posing intervention had a positive and statistically significant impact on the student learning outcomes, specifically knowledge-based, skill-based, ability-based, and attitude-beliefs. The comparisons among the learning outcomes showed that problem posing had the largest positive impact on knowledge-based, mean Hedge's g effect size of 1.31 with a 95% confidence interval of 1.04 to 1.59.

Homogeneity of Effect Sizes and Publication Bias
With Q statistic (chi-square) as a variance indicator, four studies under knowledge-based were heterogeneous wherein Q = 68.12 (df = 3, p < .001) indicating significant variability of one or more effects from the mean. In contrast, the impact of problem posing on student skill-based was large but Q = 7.06 (df = 4, p = .13) showing it www.ccsenet.org/ies International Education Studies Vol. 7, No. 13; was not statistically significant indicating the studies were homogenous and could have come from the same distribution with a common mean. Also, the results of the Q statistic showed student learning outcomes, ability-based and attitude-beliefs were affected statistically (p < .001) by problem posing with Q = 24.65 (df = 6) and Q = 71.78 (df = 5) respectively revealing significant heterogeneity in the mean effects.
We sought to examine the funnel plots of each learning outcome by drawing smooth curves that connected the confidence limits of the effect sizes as displayed in Figure 1. The first funnel plot for knowledge-based indicates two studies, one at the bottom left and the other at the top right were outside the intervals of the mean effect. This indicated these studies were different from others which were also evidenced by the result of the Q statistic. Similarly, two to three effect size indices were not inside the confidence limits of ability-based and attitude-beliefs outcome, supportive of heterogeneity in the mean effects. In contrast, with respect to skill-based learning outcome, none were heterogeneous because all the studies were inside the confidence limits. In order to delve more deeply, we used ANOVA to examine the effects of the hypothesized moderator on these learning outcomes.

Moderator Effects
The results of the homogeneity analysis showed there might be some relationships between the learning outcomes (i.e., knowledge-based, skill-based, ability-based, attitude-beliefs) with the hypothesized moderators (i.e., educational setting, presentation type, research design, publication source). The discussion of moderator analysis will be presented according to each learning outcome based on the fixed effects model.

Knowledge-Based
Results in Table 4 showed that educational settings and publication sources had different effects statistically (p < .001) on student knowledge-based, indicating that the effect sizes were related to both moderators. In addition, the Q within (Q = 4.95, p = .08) showed the effects of studies within grade 3-12 and teacher preparation groups could have come from the same distribution of effects. In contrast, the effect of publication source had heterogeneity variance overall within groups (Q = 42.93, p < .001) indicating some studies were statistically significantly different from each other. We excluded the moderator analysis for research design because these studies only used experimental and control groups, whereas the results of moderator analysis for presentation type were similar with the publication source.  Note: *only one study involved. Note: pp. = problem posing, *only one study involved.

Skill-Based
The results from the moderator analysis (see Table 5) presented the effectiveness of problem posing intervention on student skill-based learning outcomes were dependent on educational setting, publication source, and presentation type. The values of Q within demonstrated homogenous variances overall indicating that the effects of studies within each group emanates from the same distribution of effects. No moderator analysis was performed on research design because the studies utilized similar methodologies.

Ability-Based
All the hypothesized moderators had statistically (p < .001) significant effects on student ability-based as shown in Table 6. Only presentation type showed homogeneity variance overall (Q within = 4.94, p = .42) indicating the studies within structured and non-specific problem posing might come from the same distribution of effects. www.ccsenet.org/ies International Education Studies Vol. 7, No. 13;  Note: pp.= problem posing, *only one study involved.

Attitude-Beliefs
Results of the fixed effect model for student attitude and beliefs within the hypothesized moderators is displayed in Table 7. The educational setting, research design, publication source, and presentation type had significant effects statistically (p < .001) on student attitude and beliefs toward mathematics. In addition, the Q within in each category indicated heterogeneity variance overall showing some studies were significantly (p < .001) different from others. Note: pp. = problem posing, *only one study involved.

Discussion
This meta-analytic study advances current understanding of the effects of problem posing interventions on student outcomes in three valuable ways. First, it provides insights into the potential benefits of problem posing across the four coded outcomes of knowledge-based, ability-based, skill-based, and attitude and belief. Second, it provides valuable insights about the typical effect sizes one can expect for similar interventions in at least strong quasi-experimental studies. Third, the results show the power of being able to aggregate the studies based on Q statistics analyses and being able to control for publication bias in estimating the effects of problem posing on student learning.
Initially the strong mean effects for ability-based, skill-based, and attitude-beliefs learning outcomes show that problem posing provides considerable benefits, accounting for nearly the same level of effect across these learning outcomes. The moderators for these three learning outcomes (1) educational settings (grade 3-12 versus teacher preparation); (2) publication sources (published versus unpublished); and (3) presentation types are all unequivocal. The three learning outcomes perform equally across the moderators with as much benefit for grades 3-12 students as for preservice teachers, similar results for published versus unpublished sources, and across presentation types. Knowledge-based learning outcomes encompass a broad measure of mathematics achievement. The very large effect size for knowledge-based learning outcomes (1.31) is arguably the broadest measure of student success. Therefore, the case can be made that this learning outcome is possibly the most important of the three and exhibits the greatest effect across studies.
These meta-analytic results are supportive of similar results in previous individual studies (e.g., Cai, 1998Cai, , 2003Cai & Hwang, 2002;English, 1997English, , 1998Lavy & Bershadsky, 2003;Moses et al., 1990;Silver et al., 1996;Stoyanova, 1999). Further, the large positive effect sizes ranging from .76 to 1.31 provide reasonable and robust benchmarks when planning future studies, estimating sample sizes, and deciding on designs. These results are also important for classroom application and helping school leaders in decision making about implementation of a problem-posing program (cf. Lipsey & Wilson, 2001). School leaders should consider the four learning outcomes of knowledge-based, ability-based, skill-based, and attitude-beliefs, by reviewing student performance in these areas before they decide upon a problem-posing program. After due consideration of how students are performing in each area they can customize their program to the needs of students and have reasonable expectations for the effects they can expect from their efforts. This will make communication with teachers, parents and students more effective and provide a solid rationale for why they are adopting a problem-posing program.
Preservice teachers, that are students in a teacher preparation program, were strongly influenced by problem-posing interventions across all learning outcomes. This result poses its own set of interesting and thought provoking questions. First, given the power of problem posing for preservice teachers, why has it not become more prominent in the best selling mathematics education methods textbooks? Second, given the weight of evidence for skill, ability, and attitude learning outcomes for preservice teachers, why is the topic of developing problem posing not more broadly implemented to help preservice teachers foster problem posing before they transition from education student to classroom teacher? Given only one study examining knowledge learning outcomes, it is difficult to claim robustness of the effect size estimate. However, this provides an opportunity for future research and potentially for dissertations and theses. It will be important to determine if other studies with different characteristics exhibit similar effects. Finally, other factors might be involved in causing the variations in the effect sizes for the learning outcomes; however, because of the lack of information reported in each study it is not possible to completely understand all factors.
In general, it is promising to see that having people engaged in problem-posing activities can have broad potential benefits on students' mathematics achievement, problem solving skills, levels of problems posed, and attitudes toward mathematics. This is the first quantitative synthesis on the topic, this study could not be compared with other meta-analyses for problem posing in the area of mathematics education.
This meta-synthesis has several limitations. First, in terms of quality of the studies analyzed, there were few quantitative studies published on problem posing that included relevant data for calculation of effect sizes. Many studies did not provide detailed information about the problem-posing intervention that they used. The lack of www.ccsenet.org/ies Vol. 7, No. 13; description makes replication difficult and aggregation more complex. In addition, some of the studies did not clearly present the results systematically. As a result, we had some difficulties analyzing the data provided from most of the studies. In fact, we excluded studies from which we were unable to extract the information after examining them. Secondly, only a small number of quantitative studies selected for this meta-analysis included randomization into groups. Many of the studies analyzed for this study implemented quasi-experimental designs with two-groups (experimental and control) and single-group (pre-post) designs. While using a pre-post design is better than posttest only but not as good as multiple observations, the studies provided sufficient detail for analyses but the drawback associated with their inclusion is that a one-group design tends to inflate effect sizes (Lipsey & Wilson, 2001;Shadish, Cook, & Campbell, 2002). The sample size is perhaps the most limiting aspect of this meta-analysis. The reason for the limited sample size was in part due to the fact that most studies were qualitative and did not report means and standard deviations. Even with the large proportion of qualitative studies, there were more than enough quantitative studies to provide robust estimates. The problem with the quantitative studies was that too few provided both adequate descriptions to code the learning outcomes and the means and standard deviations for the instruments used. To a lesser extent studies failed to provide adequate descriptions of the measures or failed to provide access to researcher-designed instruments. In order for meta-analyses to inform what we know about a topic, researchers must provide access to or include their custom designed instrumentation in the published work so it can be used in replication studies. If not, every study will likely become an N of 1.
In conclusion, the results of this study are encouraging and suggest that the overall effects of problem posing interventions on teaching and learning of mathematics are positive and meaningful. Policy makers and educators should view this body of evidence when deciding on instructional strategies to implement in classrooms for improving students': a) knowledge, b) problem solving skills, c) abilities to pose problems, and d) attitude toward mathematics at all levels. In addition, the results may stimulate future researchers to produce systematic studies on problem posing interventions. Still, many questions are unanswered related to the magnitude of effects of problem posing interventions on the learning of mathematics. Therefore, more extensive and effective research needs to be conducted in this area to enhance understandings for mathematics educators.