Research Synthesis of Meta-Analyses of Preservice Teacher Preparation Practices in Higher Education

,


Introduction
The search for the holy grail of effective preservice higher education teacher preparation practices has persisted for nearly four decades (Korthagen, 2016;Lehr, 1981). The search has shifted back-and-forth from a focus on candidate and program inputs (Gitomer, Latham, & Ziomek, 1999) to program completer teacher competencies (Gossman, 2018;O'Flaherty & Beal, 2018) to preservice student performance (Macfarlane, 2015) to preservice teacher preparation practices (Hattie, 2011) to preschool to high school student academic achievement (Hattie, 2012).
Candidate inputs include such things as SAT and ACT scores, high school and undergraduate GPA, and other admission requirements (Casey & Childs, 2017). Program inputs include such things as standards for accreditation, an approved course of studies, and other teacher preparation program requirements (Cochran-Smith et al., 2016). Preservice teacher preparation program practices include such things as faculty mentoring and coaching (Campbell & Campbell, 1997), types of coursework (Clift & Brady, 2005), course-based preservice student learning methods (Levin, 1995), field experiences (Hollins, 2015), and clinical supervision (Burns, Jacobs, & Yendol-Hoppey, 2016b). The inputs and practices, taken together, constitute what Darling-Hammond (2006) contends are the "make-up" of teacher preparation programs that graduate well-prepared teachers.
Teacher preparation research has focused on the relationships between different types of preservice inputs or practices and different kinds of student outputs or outcomes (Zeichner, 2005). Most reviews of teacher preparation practices have been narrative or summative reviews (Cochran-Smith & Zeichner, 2006;Gitomer & Bell, 2016;Wilson, Floden, & Ferrini-Mundy, 2001) where the relationships between inputs and outputs, and practices and outcomes, are often confounded and open to different and often conflicting interpretations (Zientek, Capraro, & Capraro, 2008;Zientek, Ozel, Ozel, & Allen, 2012). This is the case primarily because the independent and dependent variables in most reviews have not been operationally defined, and the nature of the relationships between inputs and outputs, and practices and outcomes, have not been evaluated using an objective metric (e.g., effect sizes). Figure 1 shows a framework for how inputs and outputs, and practices and outcomes, would be expected to be related. According to this framework, variables in adjoining cells would be expected to be more highly related than those in the cells not directly connected. This would explain, for example, why inputs are not related to preservice student outcomes (Smith, Wageman, Anderson, Duffield, & Nyachwaya, 2019), and why preservice teacher preparation practices are only minimally related to pre-K to high school student outcomes (Dunst, Hamby, Howse, Wilkie, & Annas, 2019a). According to Goe (2006), this is the case because the types of linkages in Figure 1 have not been clearly articulated and investigated in teacher preparation research. Mitchell and King (2016) noted that teacher preparation program research should focus on the identification of "high impact teacher practices [that] lead to higher student achievement" (p. 16). This is depicted in Figure 1 in terms of the relationship between core teacher competencies and preschool to high school student academic achievement. Hauser and Kavanagh (2019) stated that developing preservice students' ability to learn to use core teaching practices is dependent upon the use of effective preservice teacher preparation practices. This is shown in Figure 1 in terms of the relationship between preservice teacher preparation practices and both preservice teacher outcomes and core teacher competencies. These particular relationships were the focus of investigation in the research synthesis described in this paper. Figure 1. Relationships between teacher preparation program inputs and practices and different types of preservice student, beginning teacher, and K-12 student outputs and outcomes

Purpose of the Research Synthesis
The research synthesis of preservice teacher preparation practices described in this paper was a meta-analysis of meta-analyses (Cooper & Koenka, 2012;Schmidt & Oh, 2013) of teacher preparation program practices research studies. The research synthesis both builds and expands upon existing meta-analyses of higher education studies (Ahn, Ames, & Myers, 2012;Hattie, 2009;Tight, 2018) by focusing specifically on the identification of high leverage preservice teacher preparation practices and their relationships with preservice student and beginning teacher outcomes. High leverage practices were defined in terms of the sizes of effects between the preservice practices and preservice student and beginning teacher outcomes and the confidence intervals for the effect sizes. Detailed descriptions of the research methodology and results are described in Dunst et al. (2019aDunst et al. ( , 2019b. The Dunst et al. (2019b) supplemental report includes 23 data tables of the results from each of the meta-analyses and studies in the research synthesis. The Dunst et al. (2019a) paper includes 15 data tables of the aggregated results from the data tables in Dunst et al. (2019b).
This paper includes additional findings to further highlight the particular preservice practices that emerged as high impact practices. The rank-order results of the sizes of effects for the relationships between different preservice teacher preparation program practices and different preservice student and beginning teacher outcomes are the focus of analysis. Patterns of results are used to identify very high impact, high impact, medium impact, and low or no impact preservice practices.

Search Strategy
A multi-tiered search strategy was used to locate meta-analyses of preservice teacher preparation practices. Searches of four electronic databases (e.g., ERIC, PsychInfo, ProQuest Central), four open-access search engines (e.g., Directory of Open Access Journals, Bielefeld Academic Search Engine), more than 100 journals, the reference sections of all retrieved papers and research reports, and the publications of noted teacher preparation program experts were conducted to locate preservice practice-preservice student and beginning teacher outcome meta-analyses. Research reports were included only if a preservice practice (e.g., inquiry-based learning) was compared to a control or comparative group (e.g., traditional classroom lectures) of study participants. Metaanalyses of one-group pretest--posttest studies were excluded from the research synthesis to avoid criticisms levied against previously completed research syntheses of meta-analyses of both preservice and in-service teacher training and preparation studies (Bergeron & Rivard, 2017;Myburgh, 2016).
Four types of research reports were included. The first were meta-analyses of preservice teacher preparation practices where effect sizes between the preservice practices and study outcomes were reported by the study investigators. The second were research syntheses of preservice teacher preparation practices where effect sizes were reported for individual studies but which needed to be meta-analyzed for this research synthesis (e.g., Alfieri, Brooks, Aldrich, & Tenenbaum, 2011). Third, in cases where no meta-analyses of particular types of preservice teacher preparation practices could be located (e.g., student teaching clinical supervision), meta-analyses of related practices in other disciplines were included in the research synthesis (e.g., Whittaker, 2004). Fourth, in instances were no higher education meta-analyses of particular types of preservice practices could be located, we included nationally representative studies of preservice teacher preparation practices where the results in those studies were meta-analyzed for the research synthesis (e.g., Early et al., 2007). The latter permitted a more comprehensive coverage of preservice preparation practices.

Search Results
The research synthesis included 118 meta-analyses and 12 other research studies of the relationships between the preservice practices and preservice student and beginning teacher outcomes. The meta-analyses and research studies are listed in Dunst et al. (2019aDunst et al. ( , 2019b. The 130 meta-analyses and research studies included between 5000 and 6000 studies and an estimated 2.5 to 3 million study participants. (Some meta-analyses and research studies did not include either or both the number of studies or number of participants where the totals were estimated based on the research reports that did include the number of studies and number of participants.) The majority of research reports were located in journal articles (81%). Other research reports were located in dissertations and theses (12%), conference presentations (6%), and unpublished papers (1%).

Teacher Preparation Practices
An iterative process was used to both identify the preservice practices that have been the focus of investigation and to classify the practices into subtypes of preservice teacher preparation practices. This was accomplished by two authors (CJD & RBH) with extensive experience in teacher preparation research and practice. The goal was to categorize the practices into functionally similar preservice student and beginning teacher preparation methods and procedures. The iterative process was repeated more than 10 times until internally consistent subsets of practices were identified.
Fourteen (14) different types of practices were found to be the focus of different meta-analyses or research investigations. These practices are shown in Table 1. The particular practices for each type of practice are the ones for which meta-analyses or research studies were located and included in the research synthesis. Research reports of more than 60 different preservice practices were examined in the research synthesis. These are listed in the right-handed column of Table 1. The operational definitions of each of the practices are included in (Dunst et al., 2019b(Dunst et al., ) (2019b. The nomenclature used by the investigators of the meta-analyses and research studies were retained for readers to be able to locate the original sources of evidence. Traditional teacher certification, National Board Certification, Teach for America Certification, alternative teacher certification In-field degree/certification In-field certification or degree; out-of-field certification or degree Coursework General education courses, subject matter courses, methods courses, first year seminars, course attendance

Methods of course delivery
Distance education courses, blended courses, personalized system of instruction courses, audio tutorial courses

Web-and technology-based instruction
Virtual reality instruction, computer-assisted instruction, information and communication technology instruction, internetbased instruction, technology-assisted instruction, intelligent tutoring instruction Course-based student learning methods Inquiry-based learning, problem-based learning, self-directed learning, note taking practices, visually-based learning, explanationbased learning Cooperative learning practices Small group learning, peer tutoring, peer instruction Faculty instructional practices Faculty coaching, faculty mentoring, faculty instructional practices, faculty feedback on student performance, student feedback on faculty instruction, consultative feedback on student ratings of faculty performance

Teaching method instruction
Simulation-based instruction, critical thinking instruction, teaching methods instruction, peer-facilitated teaching, microteaching, modeling teaching practices, mini-courses

Field experiences
Extended student teaching (10+ weeks), limited student teaching (5-9 weeks), course-based field experiences, service learning Field experience supervision Clinical supervision, performance feedback Induction and school-based mentoring School-based induction programs, school-based mentoring, workplace mentoring, workplace coaching

Preservice Student and Beginning Teacher Outcomes
A similar iterative process was used to categorize the outcome measures in the meta-analyses and research studies. The outcomes in each meta-synthesis and research study were content analyzed to identify the dependent variables that were the focus of investigation and to categorize the outcomes as either teaching quality measures or study participant outcomes other than teaching quality. Table 2 shows the categorization of the outcomes.
The teaching quality measures included both teaching behavior and preservice student and beginning teacher belief appraisals. The teaching behavior outcomes included measures of different types of instructional practices and classroom practices. Instructional practices were measured in terms of preservice student, student teacher, and beginning teacher teaching practices, teacher performance, and teacher-student interactions. The belief appraisal outcomes included preservice student and beginning teacher assessments of their abilities to enact teaching behavior and practices. These included preservice and beginning teacher self-efficacy beliefs (Arsal, 2014), preparedness beliefs (Ingvarson, Beavis, & Kleinhenz, 2007), and commitment to teaching beliefs (Heinz, 2015).
The preservice and beginning teacher participant outcome measures were subdivided into two types of outcomes: Performance measures and belief appraisals. The preservice student and beginning teacher performance outcomes included, but were not limited to, academic achievement, student teaching and job-related performance, knowledge acquisition, skill development, course grades, final exam scores, and GPA. The belief appraisal outcomes included judgments of either satisfaction with or attitudes toward faculty instruction, learning methods, course delivery, performance feedback, and other preservice practices. These types of belief appraisals all involved judgments of "others" behavior and practices in contrast to the teaching quality belief appraisals that all involved preservice and beginning teacher judgments of their own behavior and practices. Satisfaction with and attitudes towards course quality, faculty instructional practices, coaching and mentoring, methods of learning, etc.

Data Preparation and Analysis
The metrics for evaluating the relationships between the preservice teacher preparation practices and both preservice student and beginning teacher outcomes were mean difference effect sizes and the 95% confidence intervals for the average effect sizes (Cumming, 2012;Zientek et al., 2012). Most meta-analyses included the mean difference effect sizes for the relationships between one or more preservice practices and preservice student or beginning teacher outcomes. In cases where results were expressed in terms of other metrics or indices, these were converted to mean difference effect sizes using generally accepted methods (Borenstein, Hedges, Higgins, & Rothstein, 2009). In other cases, the mean difference effect sizes were computed from information available in individual research reports.

Data Interpretation
The effect sizes and 95% confidence intervals for each preservice practice-outcome measure relationship in each of the different meta-analyses and research studies were averaged using the weighted mean of means procedure in SPSS (IBM Corp, 2016). This permitted determination of the aggregated effects of each preservice practice for each of the different outcomes that were the focus of investigation in the meta-analyses and studies in the research synthesis and the range of effects as determined by the aggregated 95% confidence intervals. A 95% confidence interval not including zero indicates that an average effect size differs significantly from zero at p < .05 (Hedges, 1994).
Empirically derived effect size ranges were used for interpreting the strength of the relationships between the preservice and beginning teacher practices and the study outcomes for establishing insignificant, small, medium, large, and very large sizes of effects. Findings from meta-analyses of higher education studies of the same or similar types of practices examined in the present research synthesis were used to construct effect size ranges (Schneider & Preckel, 2017;Tomcho & Foels, 2008). The effect size parameters used for data interpretation were < .10 (no effect), .10 to .35 (small effect), .36 to .65 (medium effect), .66 to .90 (large effect), and > .90 (very large effect).
The extent to which the mean difference effect sizes were reasonable estimates for the relationships between the preservice and beginning teacher practices and study outcomes was assessed using the 95% confidence intervals for the point estimates (Cumming, 2012;Thompson, 2008;Wiernik, Kostal, Wilmot, Dilchert, & Ones, 2017). Average effect sizes with small confidence intervals indicate that a practice had similar effects on an outcome in different meta-analyses and research studies. Average effect sizes with large confidence intervals indicate a differential effect of a practice on an outcome of interest. In the latter case, individual meta-analyses were examined to determine if an effect size estimate was influenced by another variable.
The practical significance (Bakker, Cai, Kaiser, Mesa, & Van Dooren, 2019) and practical benefits (Pogrow, 2019) of the results was determined by looking for generalized patterns of findings between the preservice and beginning teacher practices and study outcomes (Rossman, Yore, Hand, & Shelley, 2009). Results were examined to determine if the same or similar classes of preservice practices (e.g., field experiences) were related to the same or similar classes of outcome measures (e.g., teaching quality).

Results and Discussion
The results are organized according to three sets of outcomes: Teaching quality, preservice student and beginning teacher performance, and preservice student and beginning teacher beliefs. The teaching quality outcomes included measures of different types of teaching practices and preservice and beginning teacher competencies. The performance outcomes included measures of knowledge, skills, and academic achievement. The belief outcomes included preservice student and beginning teacher attitudes toward and satisfaction with different types of preservice practices. Table 3 shows the rank-ordering for the relationships between the teacher preparation practices and the teaching quality outcome measures. Thirty-two of the 40 preservice and beginning teacher practices (80%) were associated with positive teaching quality outcomes as evidenced by the sizes of effects and confidence intervals not including zero. The sizes of effects, however, differed considerably as a function of the type of preservice practice and teaching quality outcome. Five different patterns of relationships were evident in the findings from the research synthesis where internally consistent sets of practices were related to the teaching quality outcome measures.

Very Large Sizes of Effect
Three of the four teacher preparation practices that were associated with very large effect sizes involved active preservice student engagement in learning to use different types of teaching and clinical practices. The sizes of effects for extended student teaching (10+ weeks) are especially relevant to the purposes of this research synthesis since this practice was associated with both better classroom quality and better teaching practices. The confidence interval for extended student teaching was, however, quite large. Close inspection of the practice-outcome relationships for these practices indicated that the confidence intervals for extended student teaching were due, in part, to differences in the use of classroom management practices by regular education student teachers (ES = 1.32) compared to special education student teachers (ES = .59). The large confidence interval for simulation-based instruction with deliberate practices appears related to the complexity of the clinical practice that was the focus of investigation (McGaghie, Issenberg, Cohen, Barsuk, & Wayne, 2011). The more complex the clinical practice, the smaller the size of effect. The results, taken together, nonetheless point to the importance of ongoing authentic learning opportunities to acquire the skills needed to engage in specific types of classroom and clinical practices (Bransford et al., 2003;Bronkhorst, Meijer, Koster, & Vermunt, 2011;Ericsson, 2006).

Large Sizes of Effect
Twelve of the preservice teacher preparation practices were associated with large effect sizes. Half of the practices involved activities to promote preservice student acquisition of teaching and instructional practices (e.g., teaching practices instruction, microteaching). Limited student teaching was also associated with student teacher use of teaching and classroom quality practices. The confidence intervals for the seven practices, except for the effects for peer instruction and limited student teaching, were reasonably small. The large confidence interval for the relationship between peer instruction and teaching practices is due to one study in the meta-analysis with an effect size that was an extreme outlier (Metcalf, 1995). The large confidence interval for the relationship between limited student teaching and classroom quality was due to the differences between general education and special education students in terms of their differential use of behavior management practices. The findings for practices having large effect sizes, taken together, indicate the importance of deliberate efforts to promote preservice and beginning teacher acquisition of core teaching practices as evidenced by the sizes of effects between practices emphasizing active preservice student and beginning teacher involvement in learning core teaching practices (Gossman, 2018;O'Flaherty & Beal, 2018).
Two of the preservice and beginning teacher practices associated with large effect sizes were related to belief appraisals. Teacher degree was associated with teacher beliefs about the effectiveness of teaching practices (Moulding, Stewart, & Dunmeyer, 2014) and workplace coaching was associated with a stronger sense of career commitment (Kinlaw, 1999). These types of belief appraisals are indicators of an intent to engage in optimal job performance (Stajkovic & Luthans, 1998).

Medium Sizes of Effect
Seven of the preservice teacher preparation practices were associated with medium effect sizes. Four of the seven practices involved some type of faculty or clinical supervisor feedback and guidance. Feedback and guidance had positive effects on teaching and clinical practices, self-efficacy beliefs, and university faculty-student interactions. The confidence intervals for these practices were all reasonably small except for supervisor feedback on preservice student clinical performance where the lower bound included zero (Kluger & DeNisi, 1996). The meta-analysts conducted extensive moderator analyses and found that positive performance feedback on student task completion was associated with larger sizes of effects compared to feedback that caused students to focus attention to themselves rather than on the clinical practice that was the target of feedback. That is, performance feedback was Two of the preservice practices (modelling teaching practices and simulated student teaching practices) were also associated with medium effect sizes. Both of the practices were related to preservice student and beginning teacher use of the instructional practices that were the focus of preservice preparation. The practice-outcome relationships for these preservice practices emphasize the importance of different types of preservice and beginning teacher learning experiences and opportunities as factors promoting the acquisition of core teaching competencies (Hauser & Kavanagh, 2019;Hollins, 2011).

Medium Sizes of Effect
Seven of the preservice teacher preparation practices were associated with medium effect sizes. Four of the seven practices involved some type of faculty or clinical supervisor feedback and guidance. Feedback and guidance had positive effects on teaching and clinical practices, self-efficacy beliefs, and university faculty-student interactions. The confidence intervals for these practices were all reasonably small except for supervisor feedback on preservice student clinical performance where the lower bound included zero (Kluger & DeNisi, 1996). The meta-analysts conducted extensive moderator analyses and found that positive performance feedback on student task completion was associated with larger sizes of effects compared to feedback that caused students to focus attention to themselves rather than on the clinical practice that was the target of feedback. That is, performance feedback was most effective when the focus of the feedback was on what a preservice student or beginning teacher did well and what was the consequence of his or her use of a practice. The results point to the importance of university faculty and clinical supervisor feedback and guidance as a factor influencing preservice student and beginning teacher use of effective teaching practices (Anderson & Radencich, 2001;Burns, Jacobs, & Yendol-Hoppey, 2016a;Vertemara & Flushman, 2017).
Two of the preservice practices (modelling teaching practices and simulated student teaching practices) were also associated with medium effect sizes. Both of the practices were related to preservice student and beginning teacher use of the instructional practices that were the focus of preservice preparation. The practice-outcome relationships for these preservice practices emphasize the importance of different types of preservice and beginning teacher learning experiences and opportunities as factors promoting the acquisition of core teaching competencies (Hauser & Kavanagh, 2019;Hollins, 2011).

Small Sizes of Effect
Nine of the preservice teacher practices were associated with small effect sizes. Six of these practices involved comparisons of teachers with different degrees. In all six comparisons, having a bachelor's or master's degree was associated with better classroom quality and better teaching practices compared to an associate's or high school degree. These results are consistent with the contention that advanced degrees are associated with the use of more effective teaching practices (Whitebook, 2003). The influence of teaching degree on teaching quality is not nearly as important as authentic learning opportunities and performance feedback and guidance as found for practices associated with medium to very large effect sizes.

No Effects
Eight of the teacher preparation practices had no discernible effects on the preservice student and beginning teacher outcomes as evidenced by confidence intervals including zero. Six of the eight practices are proxy measures for teacher preparation program inputs which have been found to not be related to teacher preparation program effectiveness (Mitchel & King, 2016;Smith et al., 2019). These practices included teacher certification, teaching degree, and extended teacher preparation.
One other practice is worthy of note. Whereas school-based induction programs were not related to teacher belief appraisals in terms of preparedness to teach, individual school-based coaching and mentoring were associated with a number of the preservice and beginning teacher belief measures. Findings from a number of recent research reviews indicate that individualized mentoring of beginning teachers as part of teacher induction stands out as an important factor influencing the transition to a teaching career (Sponner-Lane, 2017; Wang, 2019).

Preservice Student and Beginning Teacher Performance
The rank-ordering for the relationships between the preservice and beginning teacher preparation practices and the performance outcomes are shown in Table 4. Thirty-seven of the 41 teacher preparation practices (93%) were associated with positive outcomes as evidenced by average effect sizes greater than .10 and confidence intervals not including zero. Three things are noteworthy about the results. First, the sizes of effects differed considerably as a function of the type of preservice practice. Second, the effect sizes for the preservice practices-outcome relationships are almost identical to those found in other research reviews of the same or similar types of practices and the same or similar types of outcomes (Hattie, 2015;Tomcho & Foels, 2008). Third, all of the confidence intervals, with only a few exceptions, are very small, indicating that the average effect sizes are reasonable estimates for the practice-outcome relationships (Cumming, 2012;Young, Young, & Hamilton, 2014).

Very Large Sizes of Effect
Only two of the preservice practices were associated with very large effect sizes: Traditional classroom instruction plus peer instruction and class attendance. The former suggests that peer instruction provided students opportunities to reinforce classroom learning (Burke, Plunkett, & Li, 2017), and the latter suggests that class attedance, provides opportunities for obtaining important information as a foundation for academic success (Credé, Roch, & Kieszczynka, 2010). Both practices point to the role of different types of learning opportunities for mastering university course material and content contributing to academic achievement.

Large Sizes of Effect
Faculty coaching and performance feedback were each associated with large effect sizes. The effects of coaching and performance feedback are noteworthy since both practices reflect the importance of faculty-provided guidance and support as factors influencing preservice student performance achievement and skill development (Fletcher & Mullen, 2012;Scheeler, Ruhl, & McAfee, 2004). The large confidence interval for performance feedback was due http://hes.ccsenet.org Higher Education Studies Vol. 10, No. 1;2020 to differences in the types of feedback investigated in the studies in the meta-analysis of this type of preservice practice (Hatala, Cook, Zendejas, Hamstra, & Brydges, 2013).

Medium Sizes of Effect
Nine of the preservice practices were associated with medium effect sizes. The practices included a mix of active preservice student learning opportunities (e.g., inquiry-based learning, problem-based learning), technology and e-learning opportunities (e.g., computer-assisted learning, information and communication technology learning), field experiences (e.g., service learning), and beginning teacher feedback and guidance (e.g., workplace coaching). The combination of practices emphasizes the importance of different types of preservice and beginning teacher practices as sources of academic achievement and optimal job performance (Cohan & Honigsfeld, 2011;Darling-Hammond, 2006).

Small Sizes of Effect
Twenty-four of the 41 preservice practices-preservice student and beginning teacher performance outcomes (58%) were associated with small effect sizes. Several things are noteworthy about the patterns of results. First, and in contrast to the practices associated with medium effect sizes, the practices associated with small effect sizes include a preponderance of outcomes that are measures of knowledge and skill acquisition and academic performance (e.g., course grades). Second, many of the practices associated with sizes of effects ranked the highest are qualitatively different from those ranked the lowest. For example, practices that actively involve preservice students in improved performance (e.g., problem-based learning, self-directed learning, small group learning) tend to be ranked higher than practices that involve more passive types of learning (e.g., explanation-based learning, visually-based learning). Third, the particular practices that are associated with skill acquisition are ones that provide preservice students and beginning teacher authentic learning experiences and opportunities (e.g., service learning, field experiences). Fourth, the importance of guidance and feedback as a factor contributing to preservice student and beginning teacher performance is once again apparent in the pattern of results for the practice-outcome relationships (e.g., faculty mentoring, performance feedback, workplace mentoring).

No Effects
Four of the preservice practices had no discernible effects on preservice and beginning teacher outcomes as evidence by mean difference effect sizes smaller than .10 and/or confidence intervals including zero. Two of the practices (extended teacher preparation program and first-year teaching seminars) are the same types of teacher preparation program inputs found to have no effects on the teaching quality outcomes.

Preservice Student and Beginning Teacher Belief Appraisals
Belief appraisals are subjective judgments of a person's evaluation of a person, object, or event. Two of the primary belief appraisals used in preservice teacher education program research are attitudes toward and satisfaction with learning methods, faculty teaching, student teaching, etc. Attitudes include evaluative judgments of the positive or negative consequences of a target of appraisal (Maio & Haddock, 2010). Satisfaction is a student's judgment of "how well a learning environment supports academic success" (Lo, 2010, p. 48). Both types of belief appraisals are indirect or proxy measures on the perceived benefits of the focus of preservice and beginning teacher practices. Table 5 shows the ranking orderings for the relationships between the preservice practices and preservice student and beginning teacher attitudes toward and satisfaction with different types of teacher preparation program practices. Sixteen of the 21 practices were associated with small or medium effect sizes (76%), four of the 21 practices (24%) had no discernible effects on preservice student and beginning teacher beliefs as evidenced by confidence intervals including zero, and one practice (information and communication technology learning) was associated with a negative effect on satisfaction with this type of learning method.

Medium Sizes of Effect
Nine of the preservice practices were related to preservice student and beginning teacher belief appraisals. The preservice and beginning teacher practices that were associated with medium effect sizes included guidance and feedback (coaching and mentoring), learning methods (problem-based learning, small group learning), and method of instruction (simulation-based instruction, personal system of instruction). One of the practices (simulation-based instruction), however, had a confidence interval including zero. Examination of the studies in this meta-analysis showed that students were less satisfied with this type of instruction involving simple tasks and were more satisfied with this type of instruction involving difficult tasks (Cook et al., 2013). One other practice (field placement clinical supervision) was associated with attenuated anxiety (Whittaker, 2004). In this same meta-analysis, however, clinical supervision was positively related to self-efficacy beliefs (Table 3). NOTES. Confidence intervals in italics were estimated based on the mean effect size and standard deviation for the studies contributing to the effect size. R = Reversed scored.

Small Sizes of Effect
Six of the preservice practices were associated with small but statistically significant effect sizes as evidenced by confidence intervals not including zero. Five of the six practices involved either e-learning instruction (technologyassisted instruction, internet-based instruction, computer-assisted instruction) or in-vivo learning experiences (service learning, field experiences). The former all involved comparisons between e-learning instruction and traditional classroom instruction (e.g., course lectures). The latter involved comparisons between the two types of learning experiences and no in-vivo learning experiences.

Negative Sizes of Effects
Five of the preservice practices were associated with statistically significant negative effect sizes as evidenced by confidence intervals not including zero. These included type of course delivery (distance education courses, blended courses), visually-based learning, extended teacher preparation programs, and information and communication technology learning. Blended courses and extended preparation programs were associated with small effect sizes and information and communication technology learning was associated with a medium effect size. In all cases, preservice students were less satisfied with and had more negative attitudes toward these practices compared to more traditional types of practices (e.g., classroom lectures).

Conclusions
A number of generalized patterns of results (Rossman et al., 2009) emerged from the research synthesis of the relationships between the 14 different types of preservice practices that were the focus of investigation (Table 1) and the preservice student and beginning teacher outcomes (Table 2). These patterns of results were used to identify preservice practices on a continuum from very high impact practices to practices having little or no discernible effects on the preservice student and beginning teacher outcomes. These generalized patterns of results were identified by first considering the teaching quality outcomes followed by the preservice student and beginning teacher performance outcomes and then the preservice student belief outcomes. http://hes.ccsenet.org Higher Education Studies Vol. 10, No. 1; 2020 Table 6 shows the categorization of the results in terms of different degrees of preservice practice impacts. An internally consistent set of practices were associated with high or very high degrees of impact. These included both clinically rich field experiences (extended and limited student teaching) that included opportunities for deliberate practice, faculty and school-based coaching, clinical supervision and performance feedback, different types of learning opportunities to learn to teach, course-based experiential learning experiences, and cooperative learning opportunities. The patterns of results are consistent with a practice-based approach to teacher preparation (Darling-Hammond, 2014;Forzani, 2014;Hauser & Kavanagh, 2019) which emphasizes the importance of authentic preservice teacher preparation learning opportunities and experiences, faculty and clinical supervisor coaching, guidance, and feedback (Burns et al., 2016a;Dunst & Hamby, 2015), and practices that involve repeated and ongoing learning opportunities to perfect teaching abilities and practices (Bransford et al., 2003;Bronkhorst et al., 2011). NOTE. Practices in italics include the specific types of practices associated or not associated with preservice student and beginning teacher outcomes.
A cluster of six practices was associated with medium preservice teacher preparation impact. These included different types of less intense student field experiences (course-based field placements, service learning), faculty and school-based mentoring, performance feedback, simulated learning experiences (e.g., simulation-based practices for learning to teach, virtual reality instruction), student directed learning opportunities (e.g., self-directed courses), and different types of e-learning methods and practices. Close inspection of these practices, when compared to very high and high impact practices, indicates that they involve less preservice student and beginning teacher investment in terms of active involvement in acquiring the knowledge and skills needed to become a well prepared teacher (Darling-Hammond, 2006).
The preservice student and beginning teacher practices associated with little or no discernible impact include a preponderance of teacher preparation program inputs (Mitchel & King, 2016) and static measures of teacher preparation (Goldhaber & Brewer, 1996). These include teacher degree, number of education classes, type of teacher certification, type of teacher preparation program, and teacher preparation program preparatory practices. Except for the course-based learning methods and practices that had little or no impact on the teacher preparation practice outcomes, nearly all of the teacher preparation practices associated with little or no benefits are ones that tell us little about the make-up or content of the practices that were not associated with positive preservice student and beginning teacher outcomes.
The particular preservice and beginning teacher preparation practice having very high impact and high impact benefits, and to a lesser extent the practices associated with medium effects, provide a foundation for which types of experiences and opportunities are indicated for improving preservice teacher education. These practices provide a framework for preparing novice and beginning teachers' abilities to engage in effective teaching with students in preschool through high school (Hauser & Kavanagh, 2019;Hollins, 2011;O'Flaherty & Beal, 2018).

Implications for Preservice Teacher Preparation
The patterns of results are consistent with calls for practice-based preservice and beginning teacher preparation (Forzani, 2014;Hauser & Kavanagh, 2019;Jansse, Grossman, & Westbroek, 2015;Peercy & Troyan, 2017) that "focus professional preparation more directly on the enactment of teaching practices" (Forzani, 2014, p. 357) and clinically rich coaching and feedback on the use of the teaching practices (Burns et al., 2016a;Ong'ondo & Jwan, 2009). The results also point to the importance of active preservice student and beginning teacher involvement in knowledge and skill acquisition at all points-in-time during teacher preparation (Herrington & Herrington, 2017;Wright, 2011) and faculty coaching, mentoring, and feedback on this knowledge and skill acquisition (Fletcher & Mullen, 2012;Grima-Farrell, 2015;McGraw & Davis, 2017).
The types of preservice student and beginning teacher preparation practices identified as very high impact and high impact practices (Table 6) are especially indicated for graduating extraordinarily well prepared teachers (Darling-Hammond, 2006). Medium impact practices are indicated for providing preservice students with multiple experiences and opportunities to master foundational knowledge and skills. The particular practices used as part of teacher preparation would depend on course content and the knowledge and skills that are the focus of teacher education.

Limitations
Dunst et al. (2019a) describe a number of limitations to the focus, methodology, and results of the research synthesis described in this paper. These include the fact that no meta-analyses of frequently used preservice practices could be located (e.g., case-based learning), the unevenness in the rigor of the methodologies used by the meta-analysts of the primary studies, and the fact that our method to coalescing evidence was a main effects approach to data aggregation. It therefore could be the case that preservice practices having little or no impact may be important but that their relationships with preservice student and beginning teacher outcomes are indirect and mediated by other intervening variables.