Aptitude Tests and Successful College Students : The Predictive Validity of the General Aptitude Test ( GAT ) in Saudi Arabia

Aptitude tests should predict student success at the university level. This study examined the predictive validity of the General Aptitude Test (GAT) in Saudi Arabia. Data for 27420 students enrolled at Prince Sattam bin Abdulaziz University were analyzed. Of these students, 17565 were male students, and 9855 were female students. Multiple regression, logistic regression, and correlation analyses were conducted. The results show that the best predictor of student success at the university was the combination of high school GPA (HSGPA) and the National Achievement Test (NAT), as measured by cumulative GPA or by new students’ GPA. However, the GAT was the best predictor of graduation as a criterion of success. Conclusions and recommendations for future studies are provided.


Introduction
Worldwide, millions of students apply for university admission every year.Universities make critical decisions by admitting some students and rejecting others.Admission decisions that rely on one variable or criterion, such as high school grade point average (HSGPA) alone are unfair.Thus, standardized tests have become major factors in admission decisions at universities in many countries.Two types of tests are often used for this purpose, aptitude and achievement tests.Aptitude tests are "focused on measuring verbal and mathematical abilities not directly tied to the curriculum" (Koljatic et al., 2012), whereas achievement tests measure accomplishment.Achievement tests are helpful for improving performance and are based on clear curricular guidelines and important concepts for students to learn (Atkinson, 2001).
There are two main reasons for using standardized tests, particularly aptitude tests.First, it is unfair for a university to admit students based on HSGPA only due to the large variance in high schools' quality and grading flexibility.Standardized test scores are more comparable on the same scale (Evans, 2012).Aptitude and achievement tests have been adopted because admitting students who will succeed academically at an institution is an important issue for universities (O'Connor & Paunonen, 2007) and because of the growing belief that standardized tests are helpful for selecting students who are likely to succeed with consistently high grades from year to year (Evans, 2012).
Another reason for their use is the predictive validity of aptitude and achievement tests (Evans, 2012); these tests can predict students' success in the university.Bridgeman, Pollack, and Burton (2004) found that SAT scores were an important predictor of college "success", defined by the maintenance of a cumulative GPA within a certain range, for students from 41 institutions with similar high school scores.Zwick (2007) argued that although standardized tests increase universities' prediction abilities only slightly, they can be worthwhile, especially for universities that are unable to conduct interviews or review detailed documents for all applicants.However, some argue that students should be judged on their mastery of the high school curriculum (Lemann, 1999) rather than on what Atkinson refers to as "ill defined notions of aptitude".Atkinson (2001, p. 8) also states, "the movement away from aptitude tests towards achievement tests is an appropriate step for U.S. students, schools and universities".
Debate over the use of aptitude and achievement tests for admission purposes in postsecondary education has occurred worldwide, and Saudi Arabia is no exception.However, in Saudi Arabia, all students are required to take the General Aptitude Test (GAT) to apply for university admission in addition to providing their high school GPA (HSGPA).For some majors, an achievement test is also required as one of the criteria for admission.Some colleges, such as medical colleges, admit students based on consideration of a combination of their HSGPA, GAT score, and National Achievement Test (NAT) for health colleges (Bajammal et al., 2008).The formula for weighing these three scores (HSGPA, GAT, NAT) to generate a single score for each candidate for an admission decision varies from university to university.For example, King Saud University and Prince Sattam bin Abdulaziz University have four different formulas depending on the college to which the applicant desires admission.These formulas are as follows: (1) HSGPA 30%, GAT 30% and NAT 40%; (2) HSGPA 40%, GAT 30%, and NAT 30%; (3) HSGPA 60% and GAT 40%; and (4) HSGPA 50% and GAT 50%.The National Center for Assessment in Higher Education describes the GAT as "a test that measures the analytical and deductive ability of the student, it focuses on the student potential for learning aside from his proficiency in a particular subject; through measuring student's ability to: read comprehension; understand logical relationships; solve problems based on basic mathematical concepts; conclude; and measure " (retrieved Nov. 1, 2012).
Historically, studies have shown that aptitude tests (e.g., the SAT) and HSGPA account for only 25% of the variance in university GPA (Sparkman, 2012).However, Astin (1993) found that, based on the available admission data, HSGPA and standardized test scores were the two best predictors of university GPA as the criterion for success.Despite the debate over the efficacy of standardized tests, these tests "will continue to play an important factor in college admission for the foreseeable future" (Evans, 2012, p. 10).
As mentioned previously, one of the reasons for adopting aptitude tests for admission in universities is the predictive validity of these tests.Based on this importance, the key question of this study is whether the GAT can predict students' success at the university level.Alshumrani (2007) examined the predictive validity of the GAT and HSGPA in Saudi Arabia and found that the combination of both predictors explained approximately 11% of first-semester college GPA.HSGPA explained 10% of GPA variation, whereas the GAT explained only 1% (Alshumrani, 2007).

Method and Data
The main purpose of this study is to examine the GAT's predictive validity for success at the university level.Because studies have selected various criteria for success at the university level, this study examines the most common criteria for success at the university level: cumulative GPA for all students at different levels, first-semester (the period examined in this study) GPA, cumulative GPA for graduated students, and graduation.This examination is conducted in four different ways: first, by examining the GAT's ability to predict cumulative GPA for university students at all levels; second, by examining the GAT's ability to predict GPA for new university students using their first-semester GPA; third, by examining the GAT's ability to predict cumulative GPA for graduated university students only; and fourth, by examining the GAT's ability to predict who will graduate as opposed to who will fail to complete a degree program.
Data for 27420 students enrolled at Prince Sattam bin Abdulaziz University were analyzed.Data obtained from university records and identification numbers were cleared to ensure the confidentiality of information for 17565 male students and 985 female students.Linear regressions were conducted individually for the GAT, NAT, and HSGPA to examine their predictive validity for success.Multiple regression analysis was conducted to examine the predictive validity of the GAT, NAT, and HSGPA together.A logistic regression was conducted to examine how well the GAT predicted which students graduated versus which students failed to complete a degree.Additionally, correlations were conducted to examine the level of association of the GAT, NAT, HSGPA, and GPA.All of the results are reported at p=.05 unless otherwise stated.

Independent Variables
HSGPA achieved by students during the last year of high school is expressed on a scale of 100 points and is often treated as a percentage representing success.This is the reason for the term high school percentage that is used in Saudi Arabia.Two other independent variables in this study are GAT scores and NAT scores.

Dependent Variables
Four types of dependent variables are used in this study based on various criteria for success.Three types of data are used for the dependent variable in regression analyses: (1) cumulative GPA for students at all levels, including graduated students; (2) GPA for students after their first semester at the university; and (3) cumulative GPA for graduated students.The fourth dependent variable is the binary data for students' status (graduated or failed to graduate).

Results
According to Evans (2012), we can use the correlation coefficient of an exam score with the college outcome (cumulative GPA) to examine the predictive validity of this exam.The correlation coefficient of the GAT with GPA is .33,which means that only approximately 9% of the variation in GPA may be explained by the GAT.Hence, the GAT is not a strong predictor of validity.The highest correlation among these results is between the NAT and the GAT (R=547), which means that 29% (R 2 ) of the variation in the GAT is explained by variation in the NAT.Additionally, this finding may suggest that the GAT is focused on content-based knowledge, which leads to overlap with the NAT.The second highest correlation is between HSGPA and cumulative GPA in the university (R=.513).This is the highest correlation for GPA with any variable in the study.

Does GAT Predict GPA?
In this section, we analyze the predictive validity of the GAT, HSGPA, and NAT in different ways: as a one predictor model for each variable, as a combination of all three variables in one model, and as a linear combination of every combination of two predictors.All of these analyses were conducted with three different samples: new students (first-semester GPA), graduated students, and all students (cumulative GPA) (see Table 3)..342(n=11747)

Does GAT Predict First-Semester GPA?
The GAT's predicted validity for first-year GPA, in a typical procedure (Zwick, 2007), was examined using data for 4959 students after their first semester at the university.A similar analysis of students at different levels showed that the best predictor was the combination of HSGPA and NAT.This model explained 22% (R 2 =.225) of GPA variation.The HSGPA-only model explained 18% (R 2 =.183), as the best single predictor (Model 3 in Table 4).

Does GAT Predict Cumulative GPA?
By conducting hierarchical multiple regressions to examine whether GAT, NAT, and HSGPA predict GPA, we found that HSGPA was the strongest predictor in this model, with R 2 .30,F (1, 11743)=5027, p<.05.Thus, 30% of the variation in GPA could be explained by variation in HSGPA alone.Adding the NAT to the model increased R 2 by only .04,which explains 4% in addition to the 30% that can be explained by HSGPA.Adding the GAT to the model increased R 2 by only .002,which explains .2% in addition to the 34% explained by HSGPA and NAT.
When the GAT was used in the model alone as one predictor, the model was statistically significant, F (1, 11743)=1643, p<.05.However, this model explained only 12% (R 2 =.123) of the GPA variation compared with the 30% (R 2 =.30) that could be explained by HSGPA alone as one predictor in the model.If we include the three variables as predictors (HSGPA, GAT, and NAT) in the model and take the linear combination of these three variables, the explained portion of GPA variation increases to 34% (R 2 =.342).The model with three variables (HSGPA, GAT, and NAT) is slightly better, based on R 2 , than the first model with only one predictor (HSGPA).
However, the fifth model (HSGPA and NAT in the model) has better predictive ability according to the general principle of parsimoniousness; the model is as explanatory as the other model with fewer parameters (Seasholtz & Kowalski, 1993).Nevertheless, the first model also explains approximately 30% using only one predictor.In sum, according to this analysis, HSGPA is the strongest predictor of success in the university based on cumulative GPA, compared to the GAT or the NAT.

Does the GAT Predict Graduation?
To examine the GAT's ability to predict graduation as a criterion of success, we analyzed data on students who graduated (n=894) and students who were academically fired (students forced to drop out due to poor grades) (n=858).A logistic regression was conducted with the GAT as the independent variable and students' graduation status as the dependent variable (binary data, graduated or academically fired).Before entering a predictor, with the assumption of equal numbers in each category, the model (model 0) recorded 51% correct classifications.This percentage increased to approximately 61% by adding the GAT to the model as a significant predictor.In this model, the GAT (R 2 =.074) explained 7% of the variation in the variable of students' graduation status as a predictor of graduation from the university.This model fits with the acceptance of the null hypothesis of the Hosmer and Lemeshow test (p>.05)(Shah and Barnwell, 2003).The GAT was able to predict student status slightly better than HSGPA (R 2 =.06, 6%).The NAT model did not fit, so it was eliminated from this analysis, and R 2 =.02.

Discussion
This study found that a combination of HSGPA and the National Achievement Test (NAT) is the best predictor for success in the university.This result differs from Burton and Ramist's (2001) findings; these authors found that the combination of HSGPA and SAT scores was consistently the best predictor of university success across many studies.However, HSGPA is the common variable in both studies.

Predicting Success (GPA)
This study found that, compared to the GAT and the NAT, HSGPA is the strongest predictor of success when the criterion is cumulative university GPA.This result can be explained by one of two possibilities.First, universities use similar instructional methods and provide experiences similar to those students experience in high schools.
In other words, universities are an extension of the pre-university education, and most of the teaching and assessment methods used in these universities are not significantly different from those used in high schools.
Thus, teaching at the university level is an extension of the secondary stage in terms of teaching methods and the expectations of students and focuses on the same levels of goals that are highlighted in the secondary stage, such as goals that focus on the ability to recall data or information rather than higher-level goals that focus on application, analysis, or synthesis (see Bloom's taxonomy, 1956).And this interpretation is supported by Alqataee and Alharbi (2012).The other possible explanation is that the GAT as an aptitude test failed to capture and measure abilities that could help students succeed in a university setting.
In the first scenario, we assumed that universities have not raised the bar for their students, and instead continued to set goals at the lowest level and provided knowledge in the same way as the pre-university education.Therefore, because the similarities between high school education and university education are greater than the differences, it is reasonable that high school GPA is the most accurate predictor of university success.If it is true that university education is similar to pre-university education, this finding supports the claim that aptitude tests measure different abilities that might be ignored in high school tests.Therefore, the learning and assessment methods utilized in universities must be different and more profound than those used in secondary schools.In the second scenario, the GAT failed to capture and measure students' aptitude and potential to succeed in a university setting.If this finding is accurate, the National Center for Assessment in Higher Education needs to develop and make necessary modifications to the GAT to increase its predictive validity.

Predicting Graduation
This study found that the GAT is the strongest predictor of success when the criterion is to predict graduation, with slightly better predictive ability than HSGPA.This result is consistent with Burton and Ramist's (2001) conclusion after reviewing eight studies that examined the SAT's ability to predict graduation.Manski (1983, p. 15) also found a strong relationship between SAT scores and persistence in college, even among students with similar class ranks in high school (Manski, 1983).
In conclusion, HSGPA alone appears to be more able to predict university success than the GAT.Furthermore, the combination of HSGPA and NAT is the best predictor of success when university GPA is the criterion.However, the GAT is a slightly better predictor of success than HSGPA when graduation is the criterion.

Recommendations
Because aptitude tests are designed to measure students' potential abilities, which are not often measured in high school tests, the national Center of Testing should work to improve the predictive validity of the GAT to reach or exceed the predictive ability of HSGPA.Future studies need to examine the GAT's predictive validity across university majors.In addition, the high predictive ability of HSGPA highlights the need for university officials in Saudi Arabia to review and evaluate their programs to take students to different levels of knowledge and skills.Due to the gender-separated education system in Saudi Arabia, comparison studies by gender should be conducted to examine whether HSGPA has similar predictive validity for male and female students.Furthermore, predictive studies across majors should take place to examine whether GAT predictability will vary based on the student major in the university.Finally, a comparison a cross universities will be informative.

Table 1 .
Pearson correlations for students at all levels ** Correlation is significant at the 0.01 level (2-tailed).

Table 2
shows correlation results similar to Table1, where the NAT and GAT combination is recorded as the highest correlation for new students and all students without a specified level.