Assessment of Public Schools ’ Out-of-School Time Academic Support Programs with Participant-Oriented Evaluation

Using the participants-oriented approach, this study evaluated public schools’ out-of-school time academic support programs, corresponding to the corrective/enrichment stage of Bloom’s Mastery Learning Model and offered outside formal education’s weekday hours and on weekends. Study participants included 50 principals, 110 teachers, 170 students attending programs, 110 students not attending programs, and 61 parents, all selected through random sampling in a survey-model study in Istanbul, Turkey. Partial findings were the following. According to principals and teachers, programs were sufficiently introduced to target groups. Satisfaction of attending students with the teaching—learning process was sufficient, and students believed program participation increased their success in regular classes. However, program functioning presented some problems. Administrators and teachers think the no-cost programs resulted in lack of interest among students. In addition, problems of materials and transportation have not been completely solved. Similarly, offered classes and lessons’ content organization fall short of expectations. In conclusion, out-of-school time academic support programs play important roles in reducing differences among learning levels based on individual characteristics in collective or formal learning. Still, student needs should be fulfilled, and programs should be maintained. Further studies should be conducted on these programs’ integration into formal education.


Introduction
According to the Organization for Economic Cooperation and Development (OECD), national public education and a country's management of its educational system are critical factors in a country's development (Arthur, Hisrich, & Cabrera, 2012).Therefore, among every country's main goals are improving the educational system to align with changing world conditions, ensuring its capacity to compete with other educational systems globally, and even becoming an excellent educational model.Also, among countries' main goals are individual citizens' receiving a good education and maintaining their lives in good condition.However, as a result of factors related to individual differences, this is not always possible.Especially, as a result of various factors in formal collective learning, qualitative and quantitative differences may occur in learning outputs of the teaching-learning process even in the same amount of time and under the same conditions.Bloom's (1976) Mastery Learning Model concentrates on causes of individual differences in formal collective learning and how these differences can be controlled according to interests of students, schools, and society.In addition, Bloom's Mastery Learning Model aims to establish an error-free teaching system by addressing some variables that affect learning at schools already under control.
Pioneers in the Mastery Learning Model include the Winnetka Plan (1922), developed by Carleton Washburne with his colleagues and an approach called the Morrison Model (1926), developed by Henry C. Morrison at the First, Skinner's Programmed Instruction Model proposes that learners should actively participate in the learning process, they should progress according to their individual learning speed, and learning outcomes should be immediately checked (Hızal, 1976;Külahçı, 1985;Yaşar, 2005;Efendioğlu & Yelken, 2010;cited in Berk, 2012).The second is Carroll's (1989) Model of School Learning, which asserts that learners will attain a specified learning level when the needed learning time and additional learning opportunities are provided.

Figure 1. Components of mastery learning and their relations
Pre-assessment is brief and administered to identify whether students have the foundational knowledge and skills needed for success in the upcoming unit.If students do not have these, the teacher may present students with some remedial activities.If they do have them, pre-assessment is followed by instruction on the unit's content, the second component of Mastery Learning (Guskey, 2010).
Instruction.As in other modern learning models, the Mastery Learning Model requires developmentally appropriate and research-based instruction that engages all students in teaching and learning.To provide students with needed characteristics, such instruction should be multifaceted, adapted to the context, tied to students' interests and skills, and sensitive to individual differences (Guskey, 2010).
Formative assessment is administered to monitor students' progress based on the unit's learning goals after instruction begins.This assessment attempts to identify what and why students did not learn, as well as what and how well they did learn.Therefore, formative assessment should aim not only to identify the current situation, but also to provide guidance.According to formative assessment results, students failing to meet the expected mastery level are provided with corrective instruction, while students achieving the mastery level are provided with enrichment and extension activities (Guskey, 2005).
The teacher attempts to remedy learning problems identified by formative assessment through appropriate activities within the scope of corrective instruction.Because one corrective activity is not appropriate for everyone due to individual differences, activities should differ from previous instruction.Corrective/remedial activities can be conducted individually and/or in a small group based on formative assessment results.If, through formative assessment, the teacher finds that students have reached the mastery level, accordingly, enrichment activities that extend students' knowledge are offered (Guskey, 1987;2007).Subsequently, second or parallel formative assessment takes place.
Parallel formative assessment occurs not only to determine the effectiveness of corrective activities and students' level of learning, but also provides students another opportunity to demonstrate their success.In scope, the second formative assessment should be the same as the first, but should differ in questions and problems.A significant part of corrective/enrichment instruction activities corresponding to Mastery Learning's fourth stage have been conducted by private teaching institutions (Dershane) in Turkey, which has a highly examination-oriented educational system.Although Mastery Learning has not been practiced in the Turkish educational system in an organized way, private teaching institutions can be considered within the scope of corrective/enrichment activities, as in Bloom's Model, in method and in the additional learning time they provide.
In the Law on Private Educational Institutions, No. 5580, private teaching institutions are "institutions functioning with a view to preparing students for an upper school or examinations for attendance in higher education, improving them in the courses of their preference, and raising their level of knowledge" (Taskın, 2010).In the first years of their appearance, private teaching institutions generally operated as supplementary, and they became widespread along with increased demand as the Turkish Education System became dependent on examinations (ÖZDEBİR, 2016).Table 1 summarizes the quantitative change in private teaching institutions through the years since establishment of the Inter-University Student Selection and Placement Center (ÖSYM), founded in 1974, which centralized student placement in higher education (EARGED, 1998;Tansel, 2013;MEB, 2016;ÖSYM, 2016).The term "private teaching institution" was excluded from the list of private educational institutions in amended Law No. 5580, which is still in effect along with Law No. 6528, that came into force by being published in the Official Journal numbered 28941 (Article 9), dated March 14, 2014.In addition, the same law mentioned that these institutions would continue their educational activities only until September 1, 2015, when their activities would be terminated (Official Journal, 2014).
Actually, advocacy for shutting down private teaching institutions is not new.It was agreed to take private teaching institutions under control (not to open new institutions), citing their inequality of opportunity and lack of inspection during the Bülend Ulusu Government (1980Government ( -1983)), formed after the 1980 military coup d 'état. Law No. 2843, dated June 16, 1983, aimed to shut down private teaching institutions beginning August 1, 1984 (Duman, cited by Özoğlu from 1984(Duman, cited by Özoğlu from , 2011;;EARGED, 1998;Official Journal, 1983). However, Law No. 3035, dated July 11, 1984, paved the way for private teaching institutions to continue their activities (Akyüz, 2001, p. 342).
As for private teaching institutions' recent situation, the main opposition party appealed to the Constitutional Court for annulment and suspension of several articles of Law No. 6528, providing termination of activities.The Constitutional Court finalized the application on July 13, 2015, and published its reasoned decision in the Official Journal dated July 24, 2015.In the Court's decision, various private teaching institutions mentioned in Law No. 6528 (Article 9, paragraph g) contained the expression "not having the characteristics of preparation for entrance exams for secondary or higher education," which was removed because exclusion of private teaching institutions from the educational system without providing alternative out-of-school opportunities to meet preparatory needs for entrance examinations for upper school and higher education constituted disproportionate restriction of the right to education and learning (Judgment of the Constitutional Court, 2015).
Next, the Ministry of Education considered the Constitutional Court's judgment and published regulation No. 29439, dated August 8, 2015.This regulation enabled private teaching institutions to continue operating under the name "private instruction centers," which are private educational institutions offering education in a maximum of three science categories.In these, individuals develop their knowledge, skills, abilities, and experiences and spend free time in an educational environment appropriate for science categories, as determined by the Ministry of Education, according to appropriate teaching programs for their learning levels, interests, and wishes.In conclusion, private teaching institutions continue their activities along with some changes and restrictions.
However, the Ministry of Education began to open Out-Of-School Time Academic Support (OSTAS) Programs in public schools to reduce the need for private teaching institutions and to ensure spontaneous closure of such institutions when OSTAS Programs appeared.These programs began to operate in public schools and could be offered outside weekday hours of formal education and at weekends (at semester or summer vacations when necessary) beginning October 20, 2014, to eliminate differences that might occur in learning outcomes.Now, stakeholders need answers to the following questions.Do these programs meet expectations?If they do, to what extent do they do so?If a program is not assessed, we cannot know whether it has achieved its goal, so further regulation and improvement cannot be conducted.The problem this study addresses is that OSTAS Programs have not been thoroughly assessed by all stakeholders.Thus, this study's aim is comprehensively to assess OSTAS Programs in public schools through the participant-oriented approach that includes all stakeholders.
Ralph W. Tyler, "The Father of Evaluation" in the 1930s, introduced the Objectives-Oriented Evaluation Approach (Madaus & Stufflebeam, 2002;Stufflebeam & Shinkfield, 2014), which focuses on determining program objectives and the extent to which these objectives have been achieved.The objectives-oriented approach has five different evaluation models: (1) the Tyler Model, (2) the Metfessel-Michael Model, (3) the Logic Model, (4) the Goal-Free Program Model, and (5) Provus's Discrepancy Model (Fitzpatrick et al., 2004).
In the 1960s, Stufflebeam conceptualized the Management-Oriented Evaluation Approach based on a systems approach (Zhang et al., 2011), which emphasizes the importance of information in the decision-making process and aims to provide decision-makers with information (Stufflebeam & Webster, 1980;Worthen & Sanders, 1987;Hogan, 2007;cited in Berk, 2012).The management-oriented evaluation approach has two different models: the context, input, process, and product model (CIPP Model) and the UCLA Model.
In 1967, Michael Scriven conceptualized the Consumer-Oriented Evaluation Approach (Stufflebeam & Shinkfield, 2007), which helps decide curriculum or other services' effectiveness by considering views of those who benefit from output and from the product related to the program or of those whom the program affects (Scriven, 1991;Hogan, 2007).This approach is especially and commonly preferred for assessment of educational tools and materials.
The Expertise-Oriented Evaluation Approach is the oldest known and most widely used approach.Although its origin and the person who contributed most to this approach are not known for certain, its application in education dates to the 1800s, when standardized tests were administered for entrance to higher education (Fitzpartick et al., 2004).In this approach, evaluators, who are professional content experts, conduct an evaluation by using their professional expertise and existing criteria.Evaluations conducted by accrediting agencies for accreditation of institutions and by juries formed to grant an award or to confirm competency are typical examples of this approach (Lodico, Spaulding, & Voegtle, 2010;Kuzmina-Merlino & Savina, 2015;Uşun, 2016).
The participant-oriented evaluation approach stresses participants' importance in the evaluation process and emphasizes views of those directly involved in program activities (Hogan, 2007).In early 1967, various evaluation theoreticians began to criticize these approaches by stating that mechanical and insensible approaches to evaluation dominated education with the idea that evaluators do their job without setting foot in the program being evaluated (William, 2002;Fitzpartick et al., 2004).
Then, emerging from action research during the 1980s, the participant-oriented evaluation approach is based on Robert Stake's studies, which valued information obtained firsthand during evaluation (Stufflebeam & Coryn, 2014;Fitzpatrick et al., 2004).In this approach, evaluators are involved in a continuous, strong interaction with the application environment of the evaluand, and they directly observe and record behaviors subjected to evaluation.When necessary, evaluators can provide participants training in issues related to evaluation.In addition, qualitative and quantitative data can be used separately or together (Guba & Lincoln, 1981).
This study used the participant-oriented evaluation approach because not only does it include all participants related to the program, but also the idea that data obtained from those involved in or affected by the program will be more functional in practical research.

Method
The research model used here is the single survey model, a subtype of the general survey model.The research aimed to determine Out-of-School Time Academic (OSTAS) Programs' effectiveness and efficiency based on participants' views, but without any effort to influence them or the result.

Study Group
Conducted in İstanbul, Turkey, the research, using the random sampling method, involved obtaining data from three participant types, first, teachers (n = 110) and administrators (n = 50) offering the program; second, students who attend (n = 170) or do not attend the program (n = 110); and parents (n = 61) of these students.

Data Collection Tools
To collect data from participant groups, five questionnaires with open-and close-ended questions were developed by the researcher.The first, the Administrator Questionnaire, prepared for school administrators, aimed to determine administrative problems encountered during the program's entire process; the second, the Teacher Questionnaire aimed to assess activities occurring before, during, and after the program.The Attending Student Questionnaire queried attending students and aimed to determine program effectiveness.In addition, the Non-Attending Student Questionnaire determined reasons for non-attendance to the program.Finally, the Parent Questionnaire determined the efficiency level of program promotion, provision of information about the program, and the program's level of reaching its goals from parents' point of view.
These questionnaires were prepared through the following procedures: for participants, firstly, a detailed literature review was conducted, and item pools were created for each questionnaire.While preparing items, the researcher concentrated on creating clear, comprehensible items based on a single idea.Next, items were simultaneously submitted to ten teachers and five instructors who hold at least a Master's degree in the field (expert opinion).Experts assessed questions according to their necessity, explicitness, and originality.Then, items were reconstructed in line with their feedback.Face validity and content validity were also assessed through expert opinions.
In its finalized version, the Administrator Questionnaire consisted of Subsequently, using final questionnaire versions, a pilot study was conducted with a group adequately representing the target group.A reliability analysis was performed with the same groups (except parents and non-attending students) 3 weeks after the pilot study.Its reliability coefficient was 0.94.

Data Analysis
Frequency and percentage from descriptive statistics were used in quantitative data analysis.Content analysis was used to analyze answers to open-ended questions.Figures in parentheses in sentences of participants' directly quoted opinions indicated how many participants shared the same opinion.

Findings
Research findings were examined under five headings based on types of data collection tools.Findings related to headings are presented in tables and explained in detail.

Findings Regarding Administrative Issues
No doubt administrators play a large role in achieving OSTAS Program goals.Administrative support increases these programs' effectiveness, as it does all education activities.Undeniably, the quality of all activities in an institution and that institution's administrative style are very highly correlated.Administrators' views on main issues in establishing and conducting OSTAS Programs are summarized in Table 2.According to data in Table 2, sufficient administrative support is provided both at the school level and at the level of local educational authorities to promote, establish, and efficiently maintain OSTAS Programs.
Of participants, 54% (n=27) answered "yes," and 46% (n=23) answered "no" to the question "Should a specified fee be collected from students to conduct OSTAS Programs properly and make students take them more seriously?"Those who answered "yes" also answered the question "How much should each class cost per semester?"Table 3 shows these answers' distribution.Of administrators, 77.77% think that attending students' paying a program fee of 20-100 Liras per course/per semester would contribute to their taking the program more seriously and to conducting these programs properly.
Table 4 summarizes frequently encountered problems in OSTAS Programs according to administrators.Problems in obtaining teaching materials (problems regarding sources) 20

Physical conditions and time 19
Homework follow-up 15 As Table 4 shows, according to administrators (f=32), students' lack of interest and absenteeism are the most common problems in OSTAS Programs.This is followed by teachers' lack of interest in the program and in students (f = 24), problems regarding source materials for the teaching process (f = 20), physical conditions and time (f = 19), and homework follow-up.

Findings regarding Teachers Views About Teachin-Learning Process
OSTAS Programs' success in meeting expectations depends heavily on teachers' efforts because teachers have responsibility for planning, for conducting the teaching-learning process, and for taking measures to make the program effective.Teachers' views on activities taking place before, during, and after the program are summarized in Table 5.As shown in Table 5, teachers' knowledge regarding the OSTAS Programs opening process is generally good.However, teachers show considerable inadequacy in knowledge (Items 2 and 3) of program content (31% and 41%).In addition, students' lack of seriousness toward the program (Item 8) is among other research findings.Of teachers delivering classes in OSTAS Programs, 80% (n=80) believe these programs serve their purpose, while 20% (n=32) think differently.The following excerpts are examples by teachers who think programs serve their purpose: Subjects that could not be learned at school or at home are reinforced in these programs (n=15).They contribute to remedying learning gaps and preparing for exams (n=7).More questions can be solved (n=15).They serve as a reinforcer for increasing the quality (n=7).
The following excerpts are examples by teachers who think programs do not serve their purpose: Programs are not efficient enough because students do not take them seriously (n=7).There is lack of seriousness among students because the program is offered for free (n=5).Some of the students who take classes from other teachers have begun to doubt their own teacher's knowledge (n=2).Students do not give the necessary importance to the program (n=2).
Some teachers delivering OSTAS Program classes thought that students' needs should be fulfilled and programs should be maintained, even though they found that programs do not sufficiently serve their purpose.Of teachers, 94% stated that they viewed OSTAS Programs positively (f = 103), while 6% stated that they viewed them negatively (f = 7).
Teachers who looked positively at OSTAS Programs summarized their reasons as follows: Our education system is highly dependent on exams.Academic support programs are inevitable in such an exam-oriented system (n = 17).Students find an opportunity to eliminate their lack of knowledge through these programs ( 13).These programs serve as a reinforcer (n = 12).They provide students with test practice by reviewing course subjects and solving a great number of questions ( 9).They contribute to equality of opportunity ( 9).
Teachers who looked negatively at OSTAS Programs summarized their reasons as follows: These programs occupy playtime of students at this age (n = 3).They are not efficient; they lead to a great waste of time (n = 2).They have serious shortcomings; these shortcomings need to be eliminated (n = 1).
Half of teachers looked positively at the necessity of collecting fees from students to make them take OSTAS Programs seriously (n=55), while the other half looked negatively at this suggestion (n=55).Views of those who thought that students should pay a specified fee for OSTAS Programs are summarized in Table 7.As shown in Table 7, 64% of teachers suggested that students should pay 50 to 100 Turkish liras per class for the OSTAS Program.

Findings Regarding Attending Students' View on the Effectiveness of Different Program Dimensions
For determining OSTAS Programs' effectiveness, the most important evaluation data come from direct beneficiaries, that is, from students.This section summarizes students' assessments under "introduction and informing," "teaching-learning process," "outputs," and "other issues." Findings regarding attending students' views of the program on the "Introduction and Informing" dimension are summarized in Table 8.As shown in Table 8, attending students stated that the program was adequately introduced to them and that they were well informed.Their positive views ranged from 68.6% to 88.6%.However, their views of two items (Items 3 and 4) are below 50%.Therefore, students' opinions were evidently not considered while opening programs.
Students' views on the teaching-learning process are shown in Table 9. Students' views on the activities' level of meeting expectations in the teaching-learning process ranged from 68.6% to 90%.In addition, students believed that classes were not crowded.However, they stated that the qualitative and/or quantitative sufficiency of materials and the environment's physical conditions were not sufficient (Items 10 and 11).
Findings on students' views on program "Outputs and Outcome" are shown in Table 10.As shown in Table 10, attending students stated that OSTAS Programs' outcome and outputs have met expectations at average and slightly above-average levels (from 55.7% to 74.3%).However, they also expressed that subjects they could not understand at school became more comprehensible with the program's support (gain: 80%, item 7).
Findings on students' views of "Other Issues" in the programs' functioning are shown in Table 11.Of attending students, 68.6% think that lessons are efficient.Similarly, 71.4% would recommend these programs to others.In addition, 62.9% think that the program's weekly hours are sufficient.

Findings Regarding Students' Reasons for Not Registering/Not Attending the Program
Reasons a student does not benefit from an activity although s/he has the opportunity might provide important information about that activity, and the same is true for assessment of OSTAS Programs.Findings regarding students' reasons for not registering/not attending the program are summarized in Table 12.-I do not want to come to the school.-I do not want to attend classes 6 days a week.
-I take the same lessons in the dormitory.
-I do not want to come to school on Saturdays As seen from Table 12, transportation and other physical reasons stand important reason for students not to attend OSTAS Programs since they could not reach this opportunity because they could not reach it.

Parents' Views Regarding OSTAS Programs
The last group participating in the OSTAS Programs' assessment is parents of attending students.Table 13 summarizes their views.Parents of attending students were satisfied with the program's general functioning.Of parents, 96.8% participating in the research emphasized that these programs increased their students' success, while 93.5% stated that teachers showed sufficient care, which raised students' interest in school/lessons.Similarly, 90.3% stated that their children willingly attended the program, while 87.1% expressed that they were sufficiently informed about programs, and lessons in the program were those they requested.

Discussion
This study used a participant-oriented evaluation approach that considered stakeholders' views and attempted to determine the level of satisfaction with OSTAS Programs based on all stakeholders' views.
Findings from administrators indicated that administrative measures have been taken to promote these programs to stakeholders and to ensure their efficiency.This result is an effective step for ensuring elimination of demands for private teaching institutions that at first offered support in examination preparation and then, over time, became a "shadow education system" (Özoğlu, 2011).In other words, the benefit expected from private teaching institutions is largely obtained from these programs as well.This opinion was also supported by findings of Göksu & Gülcü (2016), who concluded that administrators and teachers cared about OSTAS Programs for filling gaps that occurred due to private teaching institutions' shutdown.Similarly, Ergün (2017) and Canpolat & Köçer (2017) concluded that these courses were already successful in providing equal opportunities for students and preparing them for examinations by replacing private teaching institutions.Ünsal & Korkmaz (2016) also concluded that these programs decreased demands for private teaching institutions.
The majority of administrators think a specified fee should be charged to ensure that students take OSTAS Programs more seriously.This result is supported by results obtained in previous related studies (Göksu & Gülcü, 2016;Ünsal & Korkmaz, 2016;Canpolat & Köçer, 2017;Bozbayındır & Kara, 2017;Ergün 2017).According to these studies, parents are encouraged to register their students in these programs since they are free, and that contributes to the family economy.However, that they are offered free reduces their importance and causes lack of interest and absenteeism among students.A majority of administrators (57.25%) think 50-100 Turkish liras should be charged per class to eliminate these disadvantages and to ensure that both students and parents take these programs more seriously.
According to administrators, problems encountered in OSTAS Programs fall into the following order of priority: lack of interest and absenteeism among students, teachers' lack of interest in the program and the student, problems in obtaining teaching materials, physical conditions, and homework follow-up.Similar results (problems) have been encountered in Toprakçı & Karabacak's study (2017) and in all previously mentioned studies.However, subsequent studies have observed that some problems have been eliminated.For instance, the last study conducted in Turkey by Ergün (2017) reported no problem in obtaining materials.Therefore, authorized bodies have considered results from assessments conducted both internally (by the Ministry of National Education) and externally (by universities) since 2014, so improvements have occurred over time.
Teachers' views toward OSTAS Programs are also generally positive.Teachers believe these programs serve their purpose, that is, help students earn higher scores on examinations and increase their academic success.This result is substantially compatible with findings of Mahoney, Cairns, & Farmer (2003), Mahoney, Lord, & Carryl (2005), Little, Wimer, & Weiss (2007), and White (2010).This result completely parallels views of Küçüker (2015), Ünsal & Korkmaz (2016), Canpolat, Köçer, Bozbayındır, & Kara (2017) and Dönmez, Pekcan, & Tekçe (2016).These studies, which share partial similarities, differ from this study in that some include not only academic support provided after school, but also consist of social and sports activities as well.However, some teachers think these programs do not serve their purpose.
Teachers who think these programs do not serve their purpose argue that students' needs should be fulfilled and programs should be maintained.The main reason for these teachers' arguments is "Students do not take these courses seriously due to being provided free of charge.That reduces their value and causes some problems regarding attendance; therefore, they are waste of time."This result is supported by results of Göksu & Gülcü (2016), Ergün (2017), Bozbayındır &Kara (2017), andCanpolat &Köçer (2017).These studies also concluded that OSTAS Programs' being provided free of charge poses some problems.
More than half of teachers (64%) think charging 50-100 liras would help ensure that students take OSTAS Programs more seriously.This result not only parallels administrators' views mentioned above, but is also compatible with studies conducted by Ünsal & Korkmaz (2016), Canpolat & Köçer (2017), Bozbayındır &Kara (2017), andErgün (2017).These studies also advocated a symbolic proportion of money being charged, that is, an amount parents can afford, to make students take the program seriously.Moreover, teachers are not satisfied with the additional course fee they earn in OSTAS Programs.
More than half of teachers (56%) argue that additional teacher earnings should range from 20 to 30 liras.A year after this study's data were collected, the Ministry of National Education considered these recommendations and doubled the additional fee paid in normal formal education for these programs by enacting Law No: 29327, dated April 15, 2015(Official Journal, 2015;29327).Therefore, teachers' expectations were met soon after this study's data were collected.
Attending students' satisfaction with OSTAS Programs is generally sufficient.Students stated that programs reinforced subjects they could not understand at school, they had opportunities to solve more questions, and, therefore, their success increased in formal education.This result parallels students' views in studies of Mahoney, Lord, & Carryl (2005), White (2010), andErgün (2017).In their experimental study (experimental-control group), Mahoney, Lord, & Carryl (2005) concluded that not only students' academic success, but also their motivation and attitude were higher than those of students not attending these programs.AhmadiGatab, Shayan, & Pirhayati (2012) concluded that the academic success of students participating in OSTAS Programs was significantly higher than that of students who did not participate.Mccombs, Whitaker, & Yoo (2017) performed a comprehensive analysis of studies (meta-analysis and experimental) conducted since 2000 to evaluate OSTAS Programs' effectiveness in the United States.Students attending these academic programs showed significant progress in academic achievement and in standardized tests in the first year.In addition, the gap between the academic success of students continuing the program in the second year and students not attending the program increased.The systematic review and meta-analysis conducted by Kremer, Maynard, Polanin, Vaughn, & Sarteschi (2015) concluded that academic programs not only increased students' academic achievements, but also kept students away from bad habits (e.g., drug use, aggression).Similarly, the study by Cosden, Morrison, Gutierrez, & Brown (2004) revealed that students attending programs avoided risk behaviors and improved their academic success as a result of their tight connection with the school and society.The study of Lauer, Akiba, Wilkerson, Apthorp, Snow, & Martin-Glenn (2006) shows similar results.However, some study findings contradict those above.
Shulruf's (2010) critical review and meta-analysis concluded that the academic success of students participating in these programs was higher than that of students who did not participate although no direct cause-and-effect relation was shown between the success and content of these programs.
Although students participating in the research stated their satisfaction with OSTAS Programs, some practices of offering and conducting them are open to criticism: not considering students' requests in offering classes and/or creating their content and problems in obtaining materials.These problems have also been reported in studies by Göksu & Gülcü (2016), Ünsal & Korkmaz (2016), Canpolat &Köçer (2017), andBozbayındır &Kara (2017).However, some studies state such problems do not exist.In Ergün's (2017) research, students stated that their requests for certain classes were considered and that they had no problems regarding materials during the process.Some institutions reflecting results during 3 years of experience in the process were obtained.
Parents also believe that OSTAS Programs are beneficial because they have raised students' interest in school, and thus increased their success.In addition, they state that administrators and teachers care adequately about students.Programs' being offered free was an important factor in sending their children to the program.According to White's research (2010), parents think that middle school academic support programs offered after school and on weekends positively impacted students improved academic success.In addition, parents thought these academic programs raised students' awareness of subjects they learned in regular classes.
The last, but important research participants are non-attending students; their reasons can be divided into three main headings: The first is lack of awareness/interest due to they and/or their parents not being sufficiently informed and/or authorities' not offering requested classes.The second is not believing the program will be beneficial or attendance in other teaching institutions.The last reason is boredom with constantly being at school or physical reasons such as transportation.Bozbayındır & Kara (2017) and Canpolat & Köcer (2017) also concluded that some parents were not interested in these programs.Instead, they considered private teaching institutions, which they paid, beneficial, and they experienced transportation problems.Problems were especially experienced because school buses are not adjusted to OSTAS Programs' end time and do not operate on weekends.Dönmez, Pekcan, & Tekçe (2016) also concluded that programs created a significant course load for students.

Recommendation
The following recommendations have been developed based on this study's results.
Program introduction should be administered by means that will help reach target groups who have different opportunities and characteristics.Additionally, program promotion should target stakeholders sufficiently and correctly, and programs' sustainability should be ensured.
Each institution should consider its stakeholders' characteristics and include them in the process of rigorous, inclusive planning that is sensitive to individual differences.
For these programs, rich materials should be prepared, apart from materials used in the formal education process.
In these programs, learner-centered approaches should encourage students to take responsibility for their own learning.If necessary, teachers can attend workshops for qualification in learner-centered teaching.
School bus hours should be adjusted according to program hours where necessary.School buses should also be arranged to ensure students' weekend attendance.
A contingency approach should be considered, and supervision should be conducted to eliminate negative situations (e.g., absenteeism, lack of seriousness among students) deriving from programs' being provided free of charge or other (individual/institutional) factors.In addition, follow-up studies should be conducted for each class and each teacher according to changing conditions, results should be reviewed with all stakeholders, and commonly approved solutions should be found to improve programs.
Recommendations for researchers: Conduct studies on how such activities can be integrated into existing formal education after measures have been taken to ensure that these programs meet expectations in the best way possible in light of this and similar studies' findings.
14 close-ended items and 4 open-ended questions; the Teacher Questionnaire of 13 close-ended items and 3 open-ended questions; the Attending Student Questionnaire of 33 close-ended items and one open-ended question; the Non-Attending Student Questionnaire of 4 close-ended items and 3 open-ended questions; and the Parent Questionnaire of 11 close-ended items and one open-ended question.

Table 1 .
Quantitative change in private teaching institutions since their 1974 founding in Turkey

Table 2 .
Administrators views on main issues in establishing and conducting OSTAS Programs

Table 3 .
Distribution of administrators' answers to "How much should students pay for each class per semester?"

Table 4 .
Problems frequently encountered in OSTAS Programs according to administrators

Table 5 .
Teachers views of activities taking place before, during, and after OSTAS programs

Table 6
summarizes teachers' views on "How much should teachers earn per hour in OSTAS Programs?"

Table 6 .
Teachers' views on "How much should teachers earn per hour in OSTAS Programs?"

How much should teachers earn per hour? f Valid Percent Cumulative Percent
As shown inTable 6, 56% of teachers think they should earn from 20 to 30 Turkish liras per hour, while 19% prefer 35 to 45, and 25% prefer 50.

Table 7 .
Teachers views regarding students' paying fees for each OSTAS

Table 8 .
Views of attending students regarding introduction of the program and being informed about the program

Table 9 .
Students' views on the teaching-learning process in OSTAS Programs

Table 10 .
Students' views on OSTAS Programs' outputs and outcomes

Table 11 .
Students views on "Other Issues" of attending the OSTAS Programs That programs are offered free negatively affects my attention to lessons.If a specified fee were collected, I would be more attentive in lessons.

Table 12 .
Students' reasons for not attending the OSTAS Program in İstanbul, Turkey

Reasons for not attending the program
f 1.I was not adequately informed about the program.I did not know about such a program.2. The program was not opened, although I applied.3. Program classes were not those I wanted to take.4. My family did not allow me to participate.-They do not think the program is beneficial.-Because of the transportation and/or other physical reasons 5.I think the program is unnecessary and/or useless.6.I would attend if the teacher giving my lessons offered such a program.7. I receive support from somewhere else, or I go to a private teaching institution.8.I do not want to attend the program although my family wants me to.Other …….-The program is boring/I get bored in the program.

Table 13 .
Views of attending students' parents on OSTAS Programs