The Predictive Validity of Admission Criteria for College Assignment in Saudi Universities: King Saud bin Abdulaziz University for Health Sciences Experience

Admission criteria can be used to predict Saudi student performance in college, but significant differences across several studies exists. This study explores the predictive power of admission criteria for college assignment using King Saud bin Abdulaziz University for Health Sciences as a model. Scores from high school and standardized tests were collected for 1,595 students. Data were analyzed with multinomial logistic and multivariate linear regression. A formula was generated to determine student college assignment based on their admission criteria profile. The results showed that all admission criteria were significant predictors of college assignment but accounted for only 21.1% of the variance. Based on the results of this study, admission criteria may not be reliable predictors of college assignment on their own, and additional criteria for measuring student success are needed.


Introduction
Admission criteria and policies at Saudi universities have been significantly modified over the last several years. Initially, student admission relied only on high school grade (HSG). However, in 2001, the General Aptitude Test (GAT) and Scholastic Achievement Admission Test (SAAT) were introduced, and student admission to university depended on test performance rather than HSG. Students must have a HSG, GAT, and SAAT score in order to be admitted to the universities. GAT and SAAT are both standardized tests created by the National Saudi Center of Measurement (QIYAS). GAT measures Arabic language proficiency, critical thinking, and mathematical reasoning ability whereas SAAT measures science knowledge of biology, chemistry and physics.
Saudi universities currently use a "corrected percentage" for student admission, where scores on standardized tests are weighted. For example, at King Saud bin Abdulaziz University for Health Sciences (KSAUHS), candidates must achieve a minimum score set by the admission office for all three admission criteria (HSG, GAT, and SAAT). The university adopted the following formula to calculate the "admission corrected percentage": (HSG × 0.30) + (GAT × 0.30) + (SAAT × 0.40). Students applying to the university are ranked by the admission office based on this formula, and the admission cut-off decision is determined every year based on the available admission seats set by the university. Once admitted to the university, students are placed on a unified health science track, and their grades in the first year of university determine their college assignment.
The goal of the current study is to determine whether HSG, GAT, and/or SAAT could predict students' assigned colleges after freshman year. The study is also framed within existing research on the relationship between admission criteria and student performance (Benbassat et al., 2007;Peskun et al., 2007). Such an examination and focus on prediction of college assignment is valuable as it offers foundational information on how these criteria relate to student performance in the university (Schwartz, 2004;Roberts & Prideaux, 2010;McManus et al., 2011). This study adds to existing literature on admission criteria and its predictive power by exploring Saudi local attempts to understand student performance metrics and whether admission criteria achieve the academic purposes for which it is aimed (e.g., McManus et al., 2003;Julian, 2005;Evans & Wen, 2007;Groves et al., 2007). This study also attempts to pinpoint weaknesses in the admission process and create awareness among educators, locally and globally, on whether and how these criteria are related to student performance in college ies.ccsenet.org International Education Studies Vol. 13, No. 4;(Albanese et al., 2003Searle et al., 2003;Turnbull et al., 2003;McManus et al., 2005;Parry et al., 2006;Wright & Bradley, 2010;Wilkinson et al., 2008).
The current study is unique because existing literature only uses student GPA as an outcome, while this study uses college assignment. For instance, Albishiri et al. (2012) and Al Alwan et al. (2012) did a Saudi study relating students' admission criteria with their college GPA, and they found all of the admission criteria are significant predictors. However, both found that GAT had the lowest variance of prediction. This study is needed to determine which of these criteria best predict college assignment to better understand the relationship between admission criteria and student performance. It will provide valuable information to academic institutions and educators for better planning and resource allocation.

Research Questions
This study explores whether multiple admission criteria are related to student performance. Specific research questions in this study are the following: 1) Can HSG, SAAT, and/or GAT be used to predict students' college assignment?
2) If so, what would be prediction weight of each admission criterion individually? What would be prediction weight of all of them in a statistically combined model?
3) Is there any gender pattern of similarities and differences in terms of whether and how much each admission criterion predicts college assignment?

Study Participants
The study exists at KSAUHS pre-professional program which is designed to provide students with the skills necessary to study in the English language. It also introduces students to the basic concepts of health sciences before being admitted to one of four colleges. The first year of the program is divided into two semesters. The first semester includes three courses in English language skills and two humanities courses. The second semester consists of three English courses, three basic science courses, and one humanities course. Students are assigned to colleges based on their performance throughout both semesters as well as their college preference. Potential college assignments at KSAUHS include College of Medicine (COM), College of Dentistry (COD), College of Pharmacy (COP), and College of Applied Medical Sciences (CAMS). COM generally requires a higher GPA as it is the first college choice for the majority of students, followed by COD.

Data Collection and Analysis
The study is based on a retrospective study design which is defined as a study in which the research question was determined after the data were collected. HSG, SAAT, and GAT scores, as well as college assignment, were already coded and collected at KSAUHS. A total of 1,595 students were included in this study. Female students were 621 whereas male were 974. All of them had completed their first year at the university and took all required courses.
Data were analyzed with multinomial logistic regression in SPSS statistical program to determine whether HSG, SAAT, and/or GAT as independent variables could be used to predict student college assignment as a nominal dependent variable at KSAUHS. In addition to analyzing all data together, data from each gender were analyzed separately to explore possible differences in male and female performance. Several models were created based on scale score values and binned variables comprised of those scores.
Following multinomial logistic regression, colleges were recoded based on admission frequency to generate a dependent variable that could be used in multivariate linear regression. Models were generated for each individual admission criterion to determine its amount of variance in college assignment. An additional model with all admission criteria was also generated.

Results
CAMS was the most frequently assigned college (44.4%; Table 1), likely because, unlike the other three colleges, CAMS does not have any GPA requirements for the first year of college. Therefore, prior to analysis, CAMS students were designated as the reference group to which all other colleges were compared. In order to run the multinomial logistic regression regression, several assumptions are required to get a valid result. It includes nominal level and mutually exclusive values of the dependent variable, no multicollinearities between the independent variables, and no outliers or highly influential points. These assumptions were obtained before running into the regression analysis. According to the correlation matrix, there is no multicollinearity between ies.ccsenet.org International Vol. 13, No. 4; the test results (Table 2). In addition, a statistically significant result (i.e., p < .05) in Goodness-of-fit analysis indicates a poor fit for the model meaning there is no relationship between the independent and the dependent variables. By looking at Table 3, in this study, the model is valid, which means that it is possible predict the assigned college as a dependent variable by the admission criteria as independent variables.   Table 4 shows the model fitting information for the multinomial logistics regression. The sig. column shows the significance p = 0.000, which means that the full model statistically predicts the dependent variable better than intercept-model only with 100% probability.
Multinomial logistic regression does not have an equivalent to the R-squared that is found in the ordinal linear regression (the percentage of variance of the dependent variable explained by the predictors); there are other statistics, called Pseudo R-Square values. SPSS Statistics calculates the Cox and Snell, Nagelkerke and McFadden measures. It shows that up to 24% of the variance in college assignment can be explained by HSG, GAT and SAAT (Table 5).
Furthermore, Much greater importance, however, are the results received by Likelihood Ratio tests that show the overall effect of the predictors (Table 6). It shows that all predictors are statistically significant, but GAT gives the highest -2 Log Likelihood and highest Chi-square 5 times higher than values for other predictors.  The parameter estimates in Table 7 represents coefficients of the model. It does not show the overall statistical significance of the model, like the previous table, but as there were four categories of the dependent variable, the table contains three sets of logistic regression coefficients, representing comparison to the reference category, in this study-CAMS. Based on the Parameter Estimates table, students with high HSG are more likely to get admitted to COD. However, for COM and COP, GAT is a better predictor of college assignment.
Models for predicting male and female college assignment were run separately. For females, HSG was the best predictor of college assignment across COD, COM, and COP (Table 8). For males, SAAT was a better predictor for COD admission compared to CAMS, while GAT had more predictive power for the other two colleges as it is confirmed by the Wald value (Table 9).   HSG, GAT, and SAAT were binned into six categories based on admission frequency (Table 10). COM is the most demanding college, followed by COD, COP and CAMS. Colleges were recoded based on this to generate a dependent variable that could be used in the multivariate linear regression (CAMS = 1, COP = 2, COD = 3 and COM = 4). Multiple models were run to determine how much variance in college assignment was explained by HSG, GAT, and SAAT (Table 11). GAT explained the most variance (16.5%), followed by SAAT (11.7%) and HSG (6.6%; Table 11). The multivariate model included all three predictors and explained 21.1% of the variance in college assignment (Table 12). The multivariate model was significant as seen in the ANOVA of the model (p < 0.000; Table 13), as were all coefficients (p < 0.000; Table 14).  College assignment can be predicted by the following equation: Assigned_College = -13.522 + 0.073*HSG + 0.076*GAT + 0.027*SAAT (Table 14). Assigned college can then be predicted using the following rule: If the calculated value is less than 1.5, the student will likely be assigned to CAMS; if the value is between 1.5 and 2.5, the students will be likely assigned to COP; if the value is between 2.5 and 3.5, the student will likely be assigned to COD; and if the value is higher than 3.5, the student will likely be assigned to COM.

Discussion
Results from this study show that HSG, GAT, and SAAT can be used to predict college assignment at KSAUHS. Based on pseudo R-square values and likelihood ratio tests for the multinomial logistic regression, all three admission criteria were statistically significant predictors, but GAT was the best predictor of college assignment for male and female populations together. Results from multivariate linear regression were similar; all criteria were significant, and GAT was the best predictor. Collectively, HSG, GAT, and SAAT explained 21.1% of the variance in college assignment with significant coefficients, confirming that GAT is still the best predictive.
While all admission criteria were significant predictors, factors other than these criteria comprise 79% of the variance. These factors could include English language proficiency, the extent of science background in high school, and family and socio-economic status. The lack of variance explained by these criteria is similar to other global studies predicting student performance with admission criteria (Callahan et al., 2010;James et al., 2010;Lynch et al., 2009;Evans & Wen, 2007). Some systematic reviews of these predictive studies report that unexplained variance may be as high as 70% (Ferguson et al., 2002). Additional research is needed to determine what other factors could be used to predict student performance.
Few studies on the relationship between admission criteria and student performance in the Saudi health education context exist. These studies exhibit significant variance in the predictive power of HSG and GAT and SAAT on student performance. For example, Albishri et al. (2012) found that HSG, SAAT, and GAT were all statistically predictive of students' GPA in Saudi medical schools, and this admission criteria account for 20.8% variance in student GPA. On the other hand, Al Alwan et al. (2012) found that HSG, SAAT, and GAT explained 54% of the variance in student GPA, and HSG was the best predictor. Murshid (2012) also found that HSG was the best predictor of student GPA, and SAAT was not a significant predictor. Unlike the previous three studies, Al-Rukban et al. (2010) found that only SAAT was statistically predictive of the GPA, explaining 6.5% variance in student GPA. ies.ccsenet.org International Education Studies Vol. 13, No. 4; Several factors may be contributing to the apparent contradictions in which admission criteria best predicts student performance as well as how much variance can be explained by these criteria. For example, several studies had small sample sizes. Al Alwan et al. (2012) studied a small number of students (N = 87) and found that HSG was the best predictor of student performance. This is different from the current study, which had a much larger sample size, where HSG was the worst predictor of the three admission criteria. Additional studies across larger samples are needed.
The variance in existing literature may also be due to differences in the level(s) of students in the study. For example, Al-Rukban et al. (2010) analyzed 193 students across four different years of college. While SAAT was the only predictive criterion of students' GPA, it only explained 6.5% of the variance. It offered a different perspective since it included students from different college levels to be analyzed together. Having students with such mixed levels of college likely affects the predictive variance and produces a result that is different from other Saudi studies (Albishiri et al., 2012;Al Alwan et al., 2012;Murshid, 2012).
Albishiri et al. (2012) studied a larger sample of students (N = 727) that were at the end of year 6 and from three different health sciences schools. They found that all three admission criteria were significantly correlated with student GPA. The predictors only explained 20.8% of the variance in student GPA; SAAT was the strongest predictor whereas HSG was the lowest one. Albishiri et al. (2012) appears to be the only study besides the current one with a large enough sample size within a single year of college.
Saudi universities might consider an approach similar to the university used in this study. KSAUHS bases college assignment on student performance in their first year of college, rather than using admission criteria. The current predictive power of HSG, GAT, and SAAT is insufficient to determine how students will perform in college, but the addition of the first year of the pre-professional program would add more value to determine student readiness academically and psychologically for the demands of each college.
Through two semesters of intensive study of the pre-professional program, with different courses in English, science, and humanities, students gain valuable college experience and knowledge. Using the first year performance allows university staff to assess student readiness regardless of admission criteria. Furthermore, personality and mental traits are for important for success in health sciences colleges (Sefcik et al., 2009;Jessee et al., 2006). Students, within the stressful environment of pre-professional program, are tested daily on whether they are mentally and psychologically ready for college. Obviously, if they are not capable enough to tackle the academic demand and manage themselves and their time successfully, they would fail to do well in their academic courses. Basing college assignment on first year performance may be more beneficial than attempting to assign students to colleges based on short, semi-structured interviews. Not only can universities better assign students to colleges, but students also have the necessary skills to succeed after completing their first year. Therefore, although HSG, GAT, and SAAT are related to college assignment, the addition of the pre-professional program can provide more value and lead to a fuller understanding of student's readiness for college.
The study has limitations. It looked only into three admission criteria and only within the Saudi medical context. It focused only on one medical education program at KSAUHS. It did not consider the effect of admission criteria on other field in social sciences and humanities. It would be helpful if other studies on different fields and programs are conducted to have a comprehensive review of whether Saudi admission criteria would significantly predict students' performance in college.

Conclusion
This study explored the relationship between admission criteria and student performance. When students were sampled from KSAUHS, HSG, GAT, and SAAT were significant predictors of college assignment, and GAT was the best predictor. A formula was generated for calculating students' college assignment based on multivariate linear regression. HSG, GAT, and SAAT only explained 21.1% of the variance in college assignment. Because of this, additional performance predictors are needed. The addition of KSAUHS' first year pre-professional program created more value for college assignment. This study differed from other Saudi studies as it has a larger sample size and focuses extensively on first year performance. These differences added more complexity to the current research attempts on the Saudi predictive studies of admission criteria and may be considered for decision making by researchers and policymakers alike.