A Novel Low-Cost Indicator of Student Perseverance and Its Association with College Student Academic Performance

Within the research literature investigating how student characteristics related to perseverance impact academic outcomes, leading scholars have encouraged the development of new measurements, both survey and non-survey based. We introduce here an innovative non-survey-based measurement, derived from common higher education variables, that reflects the perseverance construct. The created perseverance proxy is easily created and explainable to audiences with minimal statistical background. The variable was used to analyze academic outcomes at a mid-sized public university in the United States. The perseverance proxy strongly positively associates with academic outcomes, as measured both by GPA and odds of academic probation, in multivariate analysis across both genders. The perseverance proxy explains more of the variation in academic outcomes than any of the cognitive and financial aid variables used in the analysis. The technique for constructing the perseverance proxy is easily replicated at any college or university having data on students’ high school ranking and college admission exam scores.


Introduction
This paper introduces to the literature a variable easily computed from data commonly found in students' college application materials that is quite likely to correlate with perseverance.The computed perseverance proxy is shown to improve the estimation of academic outcomes for students at the institution studied in this research project.Given the substantial time and resource costs associated with collecting student survey information, this approach can be a valuable alternative for researchers investigating college student retention and practitioners trying to identify students at greater risk of poor academic outcomes.

The Importance of Non-Cognitive Characteristics and the Need for Non-Survey Measurements
A large literature exists investigating variables that help to predict student academic outcomes in college.One set of variables commonly found to be significant are various measures of cognitive ability, such as the Scholastic Aptitude Test (SAT) and American College Testing (ACT) tests often required for admission to U.S. colleges and universities, while another frequently significant variable is a measure of pre-college academic performance such as a high school class rank (Noble & Sawyer, 2004;Sawyer, 2013).In the past decade, however, the scholarly community has shown mounting interest in identifying student qualities beyond cognitive ability that impact performance outcomes (Duckworth & Yeager, 2015).In particular, the concept of grit, or "perseverance and passion for long-term goals" (Duckworth, Peterson, Matthews, & Kelly, 2007, p. 1087), has gained attention as a predictor of academic achievement and success.Specifically, the construct is found, even after controlling for cognitive measures, to associate with higher Grade Point Averages (GPAs) (Duckworth, et al., 2007;Scherer, Talley, & Fife, 2017;Strayhorn, 2013), to impact persistence beyond basic training and retention through West Point Military Academy graduation (Kelly, Matthews, & Bartone, 2014), and sales workforce retention and high school graduation (Eskreis-Winkler, Shulman, Beal, & Duckworth, 2014).
Perseverance or grit has been identified as a major facet of conscientiousness (MacCann, Duckworth, & Roberts, 2009) that is related to, but distinct from, other facets of conscientiousness such as self-control (Duckworth & Gross, 2014).Self-control has been found to positively associate with a wide range of desirable outcomes (de Ridder et al., 2012;Moffitt et al., 2011).Similarly, the Academic discipline subscale within the Student Readiness Inventory (Le, Casillas, Robbins, & Langley, 2005) has been shown to improve predictions of college GPA beyond that explained by models with only high school GPA and ACT scores (Kommarraju, Ramsey, & Rinella, 2013).
While researchers have found perseverance to associate with various achievement-related outcomes, concerns have been raised, however, about relying exclusively on self-administered survey results.Particularly, obtaining survey-based measures is an unwieldy process that remain vulnerable to socially desirable responding and recall bias (Duckworth et al., 2007).Surveys of self-reporting students, or of teachers about their students, can suffer from reference bias while teacher surveys also may reflect a teacher's misinterpretation of student behavior (Duckworth & Yeager, 2015).Performance task testing is an alternative to surveys, but in addition to the costs of conducting well designed performance task tests, other shortcomings may exist as well.A one-time task test can assess a student's behavior in that test environment, but it may not be representative of the student's behavioral choices in regular daily life (Duckworth & Yeager, 2015).As such, Duckworth and Yeager (2015) recommend developing new, creative measurements and advocate for a "plurality of measurement approaches" (p.245) that includes surveys, performance task testing, and other measures to be developed.In response to their call to action, this article introduces an innovative non-survey-based measurement, derived from common higher education variables, that reflects the perseverance construct.

Other Potential Control Variables
In addition to cognitive exam scores, pre-college academic performance, and perseverance-related measures, research on college student academic outcomes has found other variables that associate with students' academic performance.Examples of these variables would include gender, race, and socio-economic status (Reason, 2009;Westrick, Le, Robbins, Radunzel, & Schmidt, 2015).In addition, in the United States differences in financing obstacles across students also are known to impact academic outcomes (ACSFA, 2010).The inadequacies of current governmental grant awards relative to college costs for lower and moderate income households, and the worsening trend of this funding gap over time, is well established (ACSFA, 2010).The larger awards to lower income households are not sufficient to cover the funding gap, contributing to lower degree completion rates for academically prepared students from lower and moderate income households (see Table 12 on p. 36 of ACSFA, 2010).While the financial aid variables are in part determined by the socioeconomic status (SES) of the student's family, and SAT scores also will associate with SES (Sackett, Kuncel, Arneson, Cooper, & Waters, 2009), we expect both sets of variables to have predictive power on GPA given prior findings in the literature.Specifically, a large majority of the correlation between SAT scores and GPA has been found to be independent of SES (Sackett et al., 2009).

Hypotheses
The hypotheses to be tested in this study are as follows.
Hypothesis 1: There is a positive relationship between GPA and the cognitive ability variables.
Hypothesis 2: There is a negative relationship between the odds of a student being placed on academic probation and the cognitive ability variables.
Hypothesis 3: There is a negative relationship between GPA and the financial stress measurement variables.
Hypothesis 4: There is a positive relationship between the odds of a student being placed on academic probation and the financial stress measurement variables.
Hypothesis 5: There is a positive relationship between GPA and the constructed perseverance proxy variable.
Hypothesis 6: There is a negative relationship between the odds of a student being placed on academic probation and the constructed perseverance proxy variable.

Sample
We used institutional data for two consecutive cohorts of incoming first year college students at a mid-sized public university in the mid-Atlantic region of the United States.The class of 2017 included 2,728 students, most of whom were Caucasian (76%); more than half (58%) were female, and were 18.2 years old on average.The class of 2018 included 2,702 students, most of whom were Caucasian (72%), more than half (56%) were female, and were 18.3 years old on average.We have four semesters of data on GPA for the 2017 class and two semesters of data on GPA for the 2018 class.All data was provided by the university, with institutional review board approval of the study design.

Non-Perseverance Related Variables in the Study
Dependent variables in the study captured two aspects of student performance, GPA and Academic Probation (=1 if GPA < 2.0).
Cognitive ability is captured in the study using students' scores on both the verbal (SATV) and math (SATM) sections of the SAT exam.
Gender is controlled for by running the GPA and Academic Probation regressions separately for males and females.
Financial stress for students is measured using both government grant awards and loan awards as proxy variables for student financial stress.

Constructing the Perseverance Proxy
High school class rank percentile (HSPCT) should associate with both cognitive measures and characteristics described in the perseverance literature.For example, optimal high school rank relative performance requires self-discipline, sustained hardiness, and long-term stamina (Duckworth et al., 2007;Kelly et al., 2014;Perkins-Gough, 2013).Hence, our proxy for perseverance for the i th student is defined as: where Predicted HSPCT i is from an Ordinary Least Squares (OLS) regression of actual HSPCT on SAT verbal (SATV) and SAT math (SATM) scores across all students in a designated class (so class 2017 run separately from class 2018).
To allow for differences by gender, equation ( 1) is estimated separately for males and females.Utilizing the prediction error from a regression of HSPCT on cognitive measures as our proxy for perseverance related student characteristics has several advantages including easily accessible data, minimal computation costs, and a straightforward interpretation for less statistically informed audiences.For example, a student with a PERSEVERE value of 15 had an actual HSPCT value 15 percentile points above the HSPCT expected, whereas a student with a PERSEVERE value of -10 had an actual HSPCT value 10 percentile points below the HSPCT expected, based on their SAT scores.While PERSEVERE is not solely attributable to unobserved differences across students in perseverance related behavior--variation in the competitiveness of high schools sending students to this university and variation in students' standardized test taking skills also contribute to PERSEVERE--it nonetheless is a single variable reflecting four years of behavior in high school that should correlate with students' unobserved perseverance.Also note that in the regression models that follow HSPCT cannot be included along with SATV, SATM, and PERSEVERE due to collinearity.If HSPCT replaces PERSEVERE in these models the overall fit (adj.R2 or pseudo-R2) is similar, but the coefficient on HSPCT does not have the same straightforward perseverance related interpretation as does the coefficient on PERSEVERE.

GPA Regression Results
With four semesters of data available for the 2017 class (Fall 2013 to Spring 2015) and two semesters of data for the 2018 class (Fall 2014 to Spring 2015), analyzed separately by gender, there are 12 OLS regressions on GPA summarized in Table 1.Nested regression models of GPA are run with the first block comprising SATV and SATM, the second block with grants and loans, and the third block with PERSEVERE.As both the White and Breusch-Pagan tests heteroscedasticity, robust standard errors were used in all regressions.The findings strongly support including our proxy measure of perseverance and splitting the analysis by gender.
A one standard deviation rise in PERSEVERE, an approximate 19.5 percentile point gain in HSPCT for females relative to that expected based on SAT scores, is associated with a 0.30 to 0.35 standard deviation rise in GPA.Likewise, a one standard deviation rise in PERSEVERE, an approximate 21.5 percentile point gain in HSPCT for males relative to that expected based on SAT scores, is associated with a 0.30 to 0.43 standard deviation rise in GPA for males depending upon the semester.Hypothesis 5 is supported for both males and females.The beta coefficients on PERSEVERE are notably larger than those for the SAT and financial aid variables for both genders, and the gain in R 2 from adding PERSEVERE as the third block in the regression is substantively larger than for any of the SAT or financial aid variables.While PERSEVERE is highly significant for both genders in each of the six regressions, it exhibits both a larger beta and a larger gain in R 2 for males than for females in five of the six semesters.
The SATV and SATM variables impact GPA in the expected manner, providing support for Hypothesis 1, but again notable differences emerge across genders.SATV is statistically significant in five of six semesters for females, but in only three semesters for males.While SATM is statistically significant in all semesters for both genders, with beta values notably larger than for SATV, its importance is larger for females as the beta on SATM in the female GPA regressions is 1.22 to 2.76 times larger than in the male GPA regressions.
The grant awards variable proves to be much more useful as a proxy for student financial stress than does the loan variable.The grant awards variable is negative and statistically significant in all 12 regressions, providing evidence strongly consistent with Hypothesis 3. Its impact, as measured by beta coefficients, is comparable to the impact from SATV, although in opposite directions.The loan variable, however, is statistically significant in only one of the male GPA regressions and two of the female GPA regressions.

Academic Probation Regression Results
While the linear regression models for GPA show clear associations with our measures of financial stress and perseverance, not all modest changes in GPA have the same impact upon students.In particular, modest reductions in GPA that drop a student onto academic probation potentially have a much larger impact on retention and graduation rates than an equivalent GPA reduction that still leaves the student comfortably above a 2.0 GPA.With this in mind, the model is rerun using nested logistic regression with a dummy (0/1) academic probation outcome as the dependent variable using the same three block structure as in Table 1 with results summarized in Table 2.
The findings are consistent with Table 1.Again, note the greater importance of SATM scores for females than for males in predicting probation as higher SATM scores reduce probation odds for females in all six semester regressions, but only once for males.SATM also has notably more probation predictive value than does SATV whose odds value never differs significantly from 1.0 in any of the probation regressions.Overall the findings support Hypothesis 2 for females, but not for males.Larger grant awards as a proxy for financial stress is associated with increased odds of probation for females in all six semesters and for males in four of six semesters, evidence consistent with Hypothesis 4. Once more, loan awards do not meaningfully associate with probation odds.Our perseverance variable continues to be the most important factor, reducing the odds of probation for both genders in every semester.Hypothesis 6 is supported.

Discussion
This research was motivated by the desire to create a proxy variable for perseverance that can be constructed from variables commonly available to researchers at many institutions.Our constructed perseverance variable is easy to replicate and has a straightforward interpretation: it is a student's relative over, or under, performance on high school class rank given their expected rank based on their SAT scores.This can be replicated for most other American institutions using either students' SAT or ACT scores.Many other nations also have some type of standardized college entrance exam which researchers could use in place of the SAT or ACT, along with data on students' secondary school class ranking or GPA.This approach is generalizable to a wide range of academic institutions globally.
We find the perseverance proxy has strong predictive value for both GPA and odds of academic probation, explaining more of the variance in these outcomes than any of the SAT or financial aid variables.Our research also supports the use of easily available financial aid variables as measures of student financial stress, finding that the student grant award variable is comparable in its impact upon academic outcomes to the SAT verbal variable for both males and females.Splitting the data by gender also is very much supported as the SAT math variable has a notably larger impact upon academic outcomes for females than for males and the perseverance variable has a larger impact on GPA for males in all but one of the GPA regressions.
We encourage researchers to conduct similar analysis on other institutions in the United States and elsewhere to assess the generalizability of these findings beyond the single mid-sized public university analyzed here.While the results may vary from institution to institution, there can be genuine value in having institution-specific findings in order to better identify students potentially at risk academically prior to problems emerging.

Table 1 .
Gender Specific Nested Regressions of GPA by Class and Semester

Table 2 .
Gender Specific Nested Logistic Regressions of Odds Academic Probation by Class and Semester Note:The odds ratios for student grants and loans are per $100 change.**, & * for p-values of < .01,& < .05;all regressions executed using robust standard errors; Probation observation = 1 if its GPA < 2.0, 0 otherwise