Further Exploring Differences in Business Undergraduate Perceived Outcomes by Preferred Classroom Learning Environment

The purpose of this study was to compare business undergraduate online/hybrid course perceptions across three different preferred classroom learning environments (CLE): online, hybrid, or face-to-face (F2F). Six different perception-based outcomes were measured: easy to use technology, peer-perceived favorability of online/hybrid courses (peer-PFoOC); instructor-perceived favorability of online/hybrid courses (instructor-PFoOC); intent to recommend online/hybrid courses; institutional commitment; and persistence towards graduation. Undergraduates who were simultaneously taking at least one online or hybrid class and F2F course, i.e., mixed course delivery format, voluntarily completed an online survey. In the fall of 2017, a complete-data sample of n = 242 respondents was obtained and in the spring of 2018 the complete-data sample was n = 237. Consistent results across both samples were found for the outcomes. Undergraduates who preferred either online or hybrid CLE had significantly higher peer-PFoOC, instructor-PFoOC, and intent to recommend online/hybrid courses than students preferring an F2F environment. There were no differences between these three CLE preference groups in perceived easy to use technology, institutional commitment or persistence. As universities increase their online and hybrid course offerings monitoring student perceived outcomes between F2F and online/hybrid course sections will continue to be important.


Introduction
The Babson Survey Research Group (2017) found that for 2016, distance/online education college student enrollment had increased for the 14th straight year.Between 2012 and 2016, the number of college students studying on a campus dropped by over one million.Many universities and colleges are increasingly viewing online education as a critical component of their enrollment strategic plan (Comer, Lenaghan & Sengupta, 2015).There has been strong growth over the past several years in colleges offering online courses and degree programs, which is expected to continue (Britt, 2015).However, as Raju andSchumacker (2014-2015) noted, the percentage of college students graduating within five years in 2012 was 51.9%, which was a decrease from 54.4% in 1991.Thus, it is also important to monitor retention of undergraduates as online education expands.The purpose of this paper was to compare business undergraduate online/hybrid course perceptions across three different preferred classroom learning environments (CLE): online, hybrid, or face-to-face (F2F) on six perceived outcomes.Two separate "mixed course delivery format" samples were used, i.e., students who take both F2F and online/hybrid classes simultaneously in a semester (Blau & Drennan, 2017;Blau, Mittal, Schirmer & Ozkan, 2017;Blau, Pred, Drennan & Kapanjie, 2016).Six different perception-based outcomes were measured: easy to use technology, peer-based perceived favorability of online/hybrid courses (peer-PFoOC); instructor-based perceived favorability of online/hybrid courses (instructor-PFoOC); intent to recommend online/hybrid courses; institutional commitment; and persistence towards graduation.

Literature Review
Three different delivery modes are now used in undergraduate college courses: face-to-face (F2F), online, and blended/hybrid.The blended or hybrid delivery mode is some combination of F2F and online, for example, alternating online and F2F meetings (Arbaugh, 2014).Prior research has compared these three delivery modes in various combinations across different disciplines, with inconsistent results.Common outcomes measured included grades (e.g., examination, final course), attitudes (e.g., engagement, satisfaction), and learning.For example, Johnson and Palmer (2015) found that students did better and were more engaged in a F2F versus online linguistics course.Using a student sample in an introductory business statistics course, Haughton and Kelly (2015) found that students in the hybrid environment performed better than students in the F2F environment on the common final exam.However, Helms (2014) found that online undergraduates did poorer in a required psychology course than F2F undergraduates.Concerning other learning outcomes, e.g., learner satisfaction and learning motivation, results comparing online with F2F or hybrid with F2F have not shown consistent differences (Arbaugh, 2014).In possibly the most comprehensive single study based on over 5,000 courses taught by over 100 faculty members over a period of 10 academic terms, Cavanaugh and Jacquemin (2015) found minimal differences in grade-based student performance between online versus F2F student samples.Arbaugh (2014) has noted that typically studies do not compare all three delivery modes simultaneously.Two exceptions are recent studies by Blau and Drennan (2017) and Blau et al. (2017).In both studies, a two-sample business undergraduate mixed course delivery format research design was used.Blau and Drennan (2017) measured student perceptions across the three classroom learning environments (CLE) on four outcomes: perceived favorability of online/hybrid courses (PFoOC); intent to recommend online/hybrid courses; perceived learning; and timely graduation.Consistent results across both samples were found for three of four outcomes.Undergraduates who preferred either online or hybrid CLE had significantly higher PFoOC and intent to recommend online/hybrid courses than students preferring F2F.There were no differences in perceived timely graduation across the three CLE groups, and inconsistent results were found for perceived learning.Blau et al. (2017) found results consistent with Blau and Drennan (2017) for intent to recommend online/hybrid courses.They also found no differences on the three preferred CLE groups for either institutional commitment or persistence towards graduation (persistence).
Similar to prior research, i.e., Blau et al. (2017) and Blau and Drennan (2017), the present study research design used business undergraduates taking at least one online or hybrid class and one F2F class (mixed course delivery format sample) in a semester, and asked what CLE they were most comfortable in, i.e., online, hybrid, or F2F.However, the present study is different from prior research since it used new samples, and controlled for a different set of covariates.Three of the perceived outcomes were different than prior research, and three were the same.Identical outcomes were: intent to recommend online/hybrid courses, institutional commitment and persistence.The three different perceived outcomes were: easy to use technology, peer-PFoOC; and instructor-PFoOC.Relevant research on each perceived outcome is briefly reviewed below.Arbaugh (2005) noted that perceived usefulness and perceived ease of use were two technological variables that could be measured when evaluating online courses.Prior research (Blau et al., 2016;Blau & Drennan, 2017) found that undergraduates did not distinguish between these two variables and they used a scale that combined both into a perceived "easy to use" technology scale.This combined six-item scale had strong reliabilities (coefficient alpha) of at least .85across multiple prior samples.To date, the authors could not find prior research comparing perceived easy to use technology across online, hybrid or F2F preferred CLE.

PFoOC
Recent research (Blau et al., 2016;Blau & Drennan, 2017) has used a general four-item PFoOC to compare specific features of online/hybrid versus F2F courses, for example, video lectures versus class lectures, written discussion board versus classroom participation, and live online discussion versus face-to-face classroom discussion.It is important to directly compare these specific features to measure the "integrity" of a course (Daymont, Blau & Campbell, 2011), i.e., approximating the same content and process in an online or hybrid course as the F2F equivalent course.As noted above, Blau and Drennan (2017) found that undergraduates who preferred either an online or hybrid CLE had significantly higher PFoOC versus F2F preferred CLE.However, despite its good reliability, the four-item PFoOC measure has limitations.For example, it does not include a group project comparison item, and many online/hybrid courses require graded group projects (Hazari & Thompson, 2015).In addition, the four-item PFoOC does not measure different types of learning, such as peer (Alavi, 1994) or transfer (Hart Research Associates, 2015).Peer learning measures student perceptions about whether they are learning from other students in class (Arbaugh, 2014).Transfer learning measures student perceptions on whether they can apply present course material to future courses or work situations (Hart Research Associates, 2015).
A prior study (Blau et al., in press) used eight items to measure PFoOC across three different mixed course delivery format samples (summer, 2017; fall, 2017; and spring, 2018), and found two related but distinct sources for PFoOC comparison, a five-item instructor-related PFoOC and a three-item peer-related PFoOC.Exploratory and confirmatory factor analyses supported these two sources, and good scale reliabilities were found across each sample: instructor-PFoOC-.91 (summer), .90(fall), and .92(spring); and peer-PFoOC-.86(summer), .85(fall), and .84(spring).To date, prior research has not compared these two scales across online, hybrid or F2F preferred CLE.

Intent to Recommend Online/Hybrid Courses
Prior research (Endres, Hurtubis, Chowdhury & Frye, 2009) found that student satisfaction with faculty practices, learning practices and course materials were each positively related to intent to recommend an online course to other MBA students.Only MBA students taking online courses were sampled.As noted earlier, prior research (Blau et al., 2017;Blau and Drennan, 2017) found that mixed course delivery format sample business undergraduates who preferred either online or hybrid CLE had significantly higher intent to recommend online/hybrid courses than students preferring a F2F CLE.This same perceived outcome will be tested in this study using different business undergraduate samples.Finding a consistent result with prior research can enhance the credibility of findings with new perceived outcome variables.

Institutional Commitment and Persistence Towards Graduation
Institutional commitment involves feeling you are attending the right institution (Davidson, Beck & Milligan, 2009), while persistence towards getting a degree is synonymous with graduation intent (Davidson et al., 2009).Studies using general college student samples (e.g., Davidson et al., 2009) and meta-analysis (e.g., Robbins et al., 2004) have shown that institutional commitment has a significant positive relationship to persistence-related outcomes.A more recent study (Beck & Milligan, 2014) of 831 online students found a strong positive relationship between institutional commitment and persistence.As noted above, Blau et al. (2017) did not find that CLE preference was related to either institutional commitment or persistence.

Controlling for Background Variables
It is important to control for background variables when investigating student perceptions (Arbaugh, 2005;Reason, 2009).Controlling for background variables when investigating the relationship of preferred CLE to outcomes allows for stronger inference about the impact of CLE.Prior research (Blau & Drennan, 2017) controlled for seven covariates: age, grade point average (GPA), number of prior online/hybrid courses taken, current class load, number of hours spent preparing for class, perceived ease of use and instructor effectiveness.The present study research design controlled for the following nine covariates: gender, age, GPA, currently working, transfer status, commuter status, number of hours spent preparing for class, prior online/hybrid courses taken, and current course load.Thus although there is some control-variable overlap (e.g., age, GPA, current course load) with Blau and Drennan (2017), several new control variables that are representative of this student population were used, i.e., gender, currently working, transfer status, and commuter status.
Based on prior research, this study asked the following general research question (RQ): RQ -after controlling for nine variables, will there be differences between online versus hybrid versus F2F preferred CLE on six perceived outcomes: easy to use technology, peer-based PFoOC; instructor-based PFoOC; intent to recommend online/hybrid courses; institutional commitment and persistence towards graduation?

Sample and Procedure
Two separate undergraduate business student samples were gathered at the end of the fall 2017 and spring 2018 semesters.These samples will be referred to as fall semester and spring semester.The business school is part of a large urban state-supported university (University X) located in the Mid-Atlantic region of the United States.The fall semester and spring semester samples each represented 14-week terms.Near the end of each semester all business undergraduate students who enrolled in at least one synchronous online or hybrid course and also an F2F course were contacted by school email address and asked to voluntarily complete an online survey.Data collections were approved by the University Institutional Review Board as part of a routine program evaluation.As an incentive to complete the online survey, two prizes were offered each semester, (e.g., two Apple AirPods) -the winners to be chosen by random number lottery.Prior research has suggested that incentives can improve online survey response rates (Fan & Yan, 2010).A student could fill out a separate survey for each online/hybrid course taken in that semester, and the student's name was entered in the lottery for each.Only respondents who completed a survey were eligible to win.Survey reminders were sent one week after the initial invitation.Across the semesters, the following number of at least partially completed survey responses was collected: fall, n = 783; and spring, n = 742.A comparison of this number of responses to the total number of eligible students in each semester indicated that the response rate was approximately 20% per semester.Prior literature has shown that a lower response rate may not be evidence of survey bias (Rindfuss, Choe, Tsuya, Bumpass & Tamaki, 2015).Across both samples, over 90% of the respondents were full-time students, i.e., taking at least twelve credits/semester.

Measures
Background control variables.In each survey, nine variables were measured: Gender, Age, GPA, Currently working, Transfer status, Commuter status, Number of hours/week spent preparing for a course, Number of prior online/hybrid courses taken; and Current course load.Gender was indicated as 1 = male, 2 = female.Age was measured in yearly response categories, using 34 categories, from 1 = 18 years old or less, 2 = 19 years old….to 34 = 51 or older.GPA (cumulative) was measured in incremental tenth response categories, using 22 categories, from 1 = less than 2.0, 2 = 2.0…. to 22 = 4.0.Currently working was indicated as 1 = no, or 2 = yes.Transfer status was indicated as 1 = no (entered as a first-semester freshman), or 2 = transferred in after first semester.Commuter status was indicated as 1 = no (living on campus or in walking distance), or 2 = yes.Number of hours a week spent preparing for class (including completing course assignments) was measured from 1 = less than one hour, to a category of 7 = 8 hours or more.Number of prior online and prior hybrid courses taken were each measured from 0 to 9 or more.However, since the means (M) for the prior hybrid courses were small M = 1.89 (fall) and M = 1.91 (spring), compared to the means for prior online courses, M = 6.83 (fall) and M = 7.56 (spring), the two measures were aggregated into a number of prior online/hybrid courses taken.Current course load was measured by asking the total number of courses being taken in a semester (online, hybrid and F2F).A breakdown of responses for each of these variables is given in Table 1.
Preferred classroom learning environment.Respondents were asked "in general, what is the most comfortable classroom learning environment for you (please select one choice below):" where 1 = online, 2 = hybrid (combination of online and face-to-face classes), and 3 = face-to-face.
Outcomes.Six perceived variables were measured using multi-item measures: Easy to use technology; Peer-PFoOC; Instructor-PFoOC; Intent to recommend online/hybrid courses; Institutional commitment; and Persistence towards graduation.A seven point response scale was used, 1 = strongly disagree to 7 = strongly disagree for all items unless otherwise noted.
Based on prior research (Blau et al., 2016;Blau & Drennan, 2017) Easy to use technology (easy to use) was measured using six items, a sample item is "Blackboard/Canvas is/was easy for you to use."These items also had a "not applicable" response which was coded as missing data.This combined six-item scale had strong reliabilities (coefficient alpha) of at least .85across prior research samples.Peer-PFoOC was measured using three-items.A sample item is "compared to face-to-face class participation, the online discussion boards were…."Instructor-PFoOC was measured using five-items.A sample item is: "compared to face-to-face courses, the quality of instructor-guided learning was…" A 7-point response scale was used for these eight items, 1 = very inferior to 7 = very superior.These items also had a "not applicable" response which was coded as missing data.Blau et al. (in press) found strong reliabilities for each scale: Peer PFoOC-.85 (fall) and .84(spring), and Instructor PFoOC-.90 (fall) and .92(spring).Intent to recommend online/hybrid courses was measured using two items.A sample item is "I would recommend taking other on-line/hybrid courses in general to students."Prior research has found a good reliability using this scale (sample), i.e., Blau andDrennan (2017)-.76 (fall, 2015) and .81(spring, 2016); Blau et al. (2017Blau et al. ( )-.88 (fall, 2016Blau et al. ( ) and .87 (spring, 2017)).Institutional commitment was measured using two items.A sample item is: "I am sure University X is the right place for me."Blau et al. (2017) found reliabilities of .95(fall, 2016) and .94(spring, 2017) using this scale.Persistence towards graduation (persistence) was measured using two items.A sample item is: "I am confident that I will graduate with a degree."Blau et al. (2017) found reliabilities of .90(fall, 2016) and .89(spring, 2017) using this scale.

Data Analyses
All data analyses were done using SPSS-PC (SPSS, 2013).Using listwise deletion, missing data across all studied variables reduced the complete data sample size to n = 242 out of 783 (31%) for the fall sample, and n = 237 out of 742 (32%) for the spring sample.This deletion also included multiple submissions from the same person in each sample, to eliminate autocorrelation as a bias (Stevens, 1996).Only complete-data samples were used to test the research question.Prior to testing the research question, correlations between outcomes were checked to make sure multicollinearity was not a problem (Stevens, 1996).Assuming sufficient independence of outcomes, Multivariate Analysis of Covariance (MANCOVA) was used to initially test the research question for the fall and spring samples.The nine control variables noted in the Measures section above were used as covariates to control for their impact on the outcomes.Prior to performing multivariate analyses Box's test for equality of covariance was successfully performed (Stevens, 1996).The independent variable was preferred classroom learning environment, and the split for the complete-data fall sample (n = 242) was: online (n = 44); hybrid (n = 90) and F2F (n = 108).For the spring sample (n = 237) the split was: online spacing (n = 47); hybrid (n = 74) and F2F (n = 116).An overall multivariate test was performed.If that test was significant this allowed for testing the impact of preferred CLE on each outcome.If this test for the impact of preferred CLE on each outcome was significant, then the least-squares difference (LSD) post hoc test comparing two groups can be used (Blau et al., 2017).

Sample Characteristics
Table 1 reports the background control variables' descriptive statistics for both samples.Both samples have higher percentages of: females, currently working, non-transfer, and on campus/within walking distance respondents.Respondents had generally taken a number of prior online/hybrid courses before participating in this study.The current course load means support the high percentage of participating full-time students.By comparison, the fall 2017 demographics for main campus matriculated University undergraduates (N = 29, 732) were: 53% female, with 93% full-time.These two comparison demographics suggest that the participating samples were generally representative.

Descriptive Statistics and Correlations for Perceived Outcomes on Both Samples
Table 2 reports the means, standard deviations, reliabilities and correlations between the six perceived outcomes.Results are generally consistent across both samples, with the highest overall means on persistence (persist), followed by institutional commitment (commit), easy to use, and recommend online/hybrid courses (recommend).Good scale reliabilities (coefficient alpha), i.e., above .70(Nunnally, 1978) are shown for each scale across both samples.Although there are significant positive correlations between all perceived measures for both samples, the highest correlation is r(308) = .79for the spring sample between peer-PFoOC and instructor-PFoOC.Although high, this is below the .80threshold for multicollinearity (Stevens, 1996) and allows each outcome to be separately analyzed.Looking at other correlations, the significant positive correlations between persistent and institutional commitment for both samples is consistent with prior literature (Robbins et al., 2004).

Missing Data Results
Given the large amount of missing data for each sample, it was important to check for systematic missing data bias (Roth, 1994) prior to testing the research question.Using an independent samples t-test (missing versus complete data respondents, for the fall sample, the only significant differences were that missing data respondents were lower on easy to use technology (M = 5.41) than complete data respondents (M = 5.72), t = -2.80(494),p < .05;while the complete data respondents were lower on persistence (M = 6.34) than the missing data respondents (M = 6.52), t = 2.62(629), p < .05.For the spring sample, missing data respondents had a lower course load (M = 4.46) than complete data respondents (M = 4.67), t = -2.14(505),p < .01;were more likely to be transfer students (M = 1.51) than complete data respondents (M = 1.36), t = 3.60(603), p < .01;and were older (M = 23 years old) than complete data respondents (M = 21 years old), t = 4.81(610), p < .01.Overall, across all the variables collected, there did not seem to be a concerning systematic bias, but rather a random pattern of missing data (Roth, 1994).The larger sample sizes helped to make smaller mean differences significant.
Thus overall, there is partial support for the research question.

Discussion
The study samples were comprised of a mixed course delivery format sample, i.e., business undergraduates simultaneously taking F2F as well as synchronous online or hybrid courses in a semester.This allowed undergraduates to directly compare the preferred CLE across the delivery modes.It is worth noting that almost 45% of the fall (108/242) and 49% of the spring (116/242) samples preferred the F2F CLE.Online or hybrid classes may not be for everyone.How can a university or college encourage F2F CLE-preference undergraduates to voluntarily "try" an online or hybrid class?Offering "ease of technology use" seminars at convenient times and locations for students, and using the application of social networking sites such as Facebook as a learning/teaching tool is one way (Wang, Lit Woo, Lang Quek, Yang & Liu, 2012).In this study, finding non-significant differences on perceived ease of using technology across the three CLE groups is an indication that a student is ready to take an online or hybrid class.The transition for an undergraduate used to taking only F2F courses may be easier if starting with a hybrid course, as opposed to an online course, since a hybrid has some F2F class meetings.The Student Online Learning Readiness (SOLR) scale (Yu & Richardson, 2015) can also help to determine undergraduate readiness for online or hybrid class learning.
Finding non-significant CLE results for institutional commitment and persistence towards graduation is consistent with prior research (Blau et al., 2017).Such results are reassuring and suggest that, although F2F may be the preferred CLE for a mixed delivery course format sample, online and hybrid course preferred CLE delivery does not lower student institutional commitment or persistence.These two outcomes are important to continually monitor, as online education offerings continue to increase (Babson Survey Research Group, 2017), while the percentage of college students graduating within five years continues to decline (Raju & Schumacker, 2014-2015).Consistent with prior research by Blau and Drennan (2017) and Blau et al., (2017), this study found that undergraduate business students preferring either an online or hybrid CLE were more likely to recommend online/hybrid courses to others versus students preferring F2F.All three outcomes, recommend, commitment and persistence were successfully measured using two-item measures.Shorter reliable measures may be useful when there are survey length constraints (Fan & Yan, 2010).
PFoOC has been previously studied as a four-item measure (Blau et al., 2017;Blau & Drennan, 2017).More recent research (Blau et al., in press) has found that an expanded eight-item PFoOC measure can be divided into two related but distinct sources, peer-PFoOC and instructor-PFoOC.An expanded measure allows for more specific item comparisons between online/hybrid to F2F classes, to better ensure course integrity (Daymont et al., 2011), i.e., approximating the same content and process in an online or hybrid course as the F2F equivalent course.Each source was found to be a reliable scale in this study across both samples.Across both samples, online and hybrid preferred CLE groups had higher peer-PFoOC and instructor-PFoOC than the F2F preferred CLE group.When comparing the means of instructor-PFoOC to peer-PFoOC, using a paired sample t-test, for each sample, the instructor-related mean was significantly higher than the peer-related mean, fall, t(241) = 7.16, p <.01, and spring, t(236) = 9.49, p <.01.This suggests that, not surprisingly, the instructor continues to play the most important role in the perceived favorability of an online/hybrid course.Helping faculty to feel as comfortable as possible with their technical skills (e.g., leading a WebEx session, developing a high-quality video lecture) is important as they prepare to teach in a virtual environment (Wingo Ivankova, & Moss, 2017).Providing necessary resources, not just technological but also full-time support staff, e.g., instructional designers, video production specialists, technology support specialists, is also critical.To be fair, this study did not control for the content of the online/hybrid course, and it is possible that peer-based learning could be higher in some courses (e.g., advanced, qualitative) versus others (e.g., introductory, quantitative).Neither course size nor instructor were controlled for in this study.Ideally, to best study PFoOC, an experimental design would be used, with students randomly assigned to either an online, hybrid or an F2F section of the same-size course.All sections would also be taught by the same instructor.This would allow for the strongest comparison of all three groups' perceived preferred CLE.However, such a research design was not possible, and some elements, particularly randomly assigning students, not would be possible to carry out.

Limitations and Directions for Future Research
There were lower cell size frequencies, especially for the online preferred CLE.Both samples were business undergraduates at a large urban state university.Testing this study's results using other samples, e.g., private, small college, non-business undergraduates, is important to see if there is generalizability.Since the research design was cross-sectional, no causal inferences can be made about relationships.All data were self-reported.A one-factor test (Podsakoff, Mackenzie, Lee & Podsakoff, 2003) showed that for the fall sample, six factors had eigenvalues of at least one, and that the first factor accounted for 18% of variance.For the spring sample, five factors with eigenvalues of one were also found with the first factor accounting for 19% of variance.This indicates that method variance, while an issue, is not an overriding limitation.Future research collecting non-self-reported data, e.g., examination or final course grades, would be helpful Haughton & Kelly, 2015;Helms, 2014).Despite the use of incentives, there was a large loss in complete-data sample size, although the missing data appeared to be random (Roth, 1994).Trying to get greater faculty involvement within each class, encouraging student participation, may boost the response rate (Nulty, 2008).This study did not measure the impact of instructor-related variables, e.g., what level of online course training did the professor have?; or how many times had the professor taught an online or hybrid course?Another perspective concerns the perception of employers who may hire students taking online courses, e.g., do they have a different view of online versus F2F classes?A research design study triangulating student-perceptions, instructor-perceptions, and employer-perceptions of online versus F2F variables (e.g., types of learning), while daunting, would be ideal in future research.

Conclusion
Online education is here to stay.Online courses can offer students a way to conveniently take college classes without having to make the trip to campus.Online courses can also save a university money (e.g., less physical classroom space needed), and such savings can be passed onto students (e.g., lower tuition).As online education continues to grow, ongoing research evaluating student perceptions comparing online/hybrid versus F2F courses is needed to assure course integrity across delivery modalities.The separate peer-PFoOC and instructor-PFoOC scales make additional specific item comparisons between online/hybrid and F2F classes.The impact of offering increased online and hybrid course sections not just on perceived student outcomes, but actual outcomes, e.g., graduation rates and retention, needs to be further studied.

Table 1 .
Descriptives for background control variables used as covariates-fall 2017 and spring 2018