Construct Validity of the Two-Factor Revised Learning Process Questionnaire in a Singapore High School

The Revised Two-factor Learning Process Questionnaire (R-LPQ-2F) is an instrument for assessing students’ learning approaches at the high school level. The instrument has significant potential for use in Singapore schools, but as yet, has not been validated in this context. This study evaluated the validity attributes of the R-LPQ-2F in a sample of Singapore senior high school students. The sample comprised 455 Year 11 students (266 male, 189 female) from Singapore. The internal structure of the R-LPQ-2F was evaluated by replicating the confirmatory factor analyses published in previous validations of the instrument, and assessing its internal consistencies and inter-scale correlations. Relationships between the R-LPQ-2F subscales and external variables were also evaluated. Results indicated that for the Deep Approach scale, a one-factor model fit the data well. For the Surface Approach scale, a four-factor model (Fear of Failure; Aim for Qualification; Minimizing Scope of Study; and Memorization) was found to fit the data best. Correlations between scores on the R-LPQ-2F subscales, on the Motivated Strategies for Learning Questionnaire, and a physics achievement test demonstrated expected patterns of correlation. Overall, results obtained in this study supported the construct validity of the R-LPQ-2F for use with Singapore high school students.


Introduction
Recent decades have witnessed a growing interest in the approaches that students adopt when undertaking their learning tasks (Biggs, 1999;Marton, Hounsell & Entwistle, 1997;Marton & Säljö, 1976).In general, research in this field has focused on two distinct types of learning approach: deep and surface (Biggs & Tang, 2007;Entwistle, 1988;Marton & Säljö, 1976).Students who adopt a deep approach to learning are depicted as those who focus on internalizing new information, connecting new understandings with existing knowledge, and analyzing how new concepts can be applied across different situations.In contrast, students who adopt a surface approach to learning are depicted as those who focus on memorizing and reproducing information, paying less attention to understanding or applying what they have learned.
Previous studies have indicated a positive relationship between deep approaches to learning and cognitive development (e.g., Nelson Laird et al., 2007;Phan, 2007Phan, , 2011)).In a recent large-scale study involving 22,000 students across 30 high schools in the United States, it was found that students who adopted a deep approach to learning demonstrated higher levels of content mastery and problem-solving skills than those who adopted more surface-based approaches (O'Day & Garet, 2014).Other research has also indicated that students who adopt a deep approach to learning tend to report higher levels of enjoyment and fulfilment in the process of learning than those who adopt a surface approach (e.g., Biggs & Tang, 201;Ramsden, 2003).skills (Goh, 1997).This remained an emphasis in the Singapore MOE's (2006) Teach Less Learn More initiative, as well its more recent (2010) 21 st Century Competency Framework (Rajah, 2013).
In order to monitor the efficacy of such initiatives, valid and reliable measures of students' learning approaches are needed.At the high school level, the Learning Process Questionnaire (LPQ: Biggs, 1987) has been used for such a purpose (e.g., Campbell et al., 2001;Cano, 2005).The original LPQ comprised three main learning approach scales: deep, surface, and achieving.The term 'achieving approach' was used to refer to strategies adopted by students to maximize academic performance (Biggs, 1987).In a subsequent revision of the LPQ (Kember, Biggs & Leung, 2004), the achieving scale was removed due to its significant overlap with the deep and surface approach scales.This revision, labelled the Revised Two-Factor Learning Process Questionnaire (R-LPQ-2F), also included a reduced number of items to make the instrument more practical to administer in the classrooms.
Each of the two main R-LPQ-2F scales (Deep Approach and Surface Approach) comprises a motive and a strategy subscale, producing four subscales in all (Deep Motive, Deep Strategy, Surface Motive and Surface Strategy).Within each subscale, there are two further components.In total, the R-LPQ-2F comprises 22 item statements, to which students respond on a five-point scale ('never or only rarely true of me' to 'always or almost always true of me').The score for each scale, subscale and component are typically computed by taking the average of their corresponding item scores.Thus, the scores range from 1 (never or only rarely true of me) to 5 (always or almost always true of me).(1) I find that at times studying makes me feel really happy and satisfied.
(5) I feel that nearly any topic can be highly interesting once I get into it.
(9) I work hard at my studies because I find the material interesting.

Commitment to Work
(13) I spend a lot of my free time finding out more about interesting topics which have been discussed in different classes.
(17) I come to most classes with questions in mind that I want answering.
(19) I find I am continually going over my school work in my mind at times like when I am on the bus, walking, or lying in bed, and so on.(3) I am discouraged by a poor mark on a test and worry about how I will do on the next test.
(7) Even when I have studied hard for a test, I worry that I may not be able to do well in it.

Aim
for Qualification (11) Whether I like it or not, I can see that doing well in school is a good way to get a well-paid job.
(15) I intend to get my A Levels because I feel that I will then be able to get a better job.

Surface Strategy
Minimizing Scope of Study (4) I see no point in learning material which is not likely to be in the examination.
(8) As long as I feel I am doing enough to pass, I devote as little time to studying as I can.There are many more interesting things to do.
(12) I generally restrict my study to what is specifically set as I think it is unnecessary to do anything extra.
(16) I find it is not helpful to study topics in depth.You don't really need to know much in order to get by in most topics.

Memorization
(18) I learn some things by rote, going over and over them until I know them by heart.
(20) I find the best way to pass examinations is to try to remember answers to likely questions.
(22) I find I can get by in most assessment by memorising key sections rather than trying to understand them.
Despite the potential utility of the R-LPQ-2F for assessing students' learning approaches, only two studies have been published to date on its psychometric properties at the high school level.The first was published by the authors of the revised version (Kember et al., 2004), and was based on a sample of 841 Hong Kong senior high school students.Results of the first-order confirmatory factor analyses performed in this study are presented in Table 2.Only models using item scores as observed variables are included here, as those using component scores as observed variables can produce very different results (e.g., Bandalos & Finney, 2001).As indicated, fit indices for the two-factor model (i.e., Model I: Deep Approach and Surface Approach) were adequate.Separate confirmatory factor analyses (CFAs) were also conducted for the four components of the Deep Approach and for the four components of the Surface Approach (i.e., Models IIa and IIb in Table 2, respectively), and both produced excellent fit indices.While the Cronbach's αs for the overall Deep and Surface Approach scales were acceptable (0.82 and 0.71 respectively), values for the individual components within these scales were low, ranging from 0.48 to 0.70.Five of the eight components fell below the commonly accepted 0.60 level (Kline, 2000).The internal consistencies of the four subscales, however, ranged from 0.58 to 0.75, with only one (i.e., Surface Motive) falling just below the 0.60 level.The latter result was attributed to the multidimensionality of this particular subscale (Biggs, 1993).The only other validity study of the R-LPQ-2F that could be located which involved a high school sample was conducted by Phan and Deo (2007).They replicated Kember et al.'s (2004)  With only two validity studies conducted on the R-LPQ-2F using high school samples, it is clear that more research is needed to evaluate its psychometric properties.The R-LPQ-2F has also not yet been validated in the Singapore context.Though Kember et al.'s (2004) study was conducted in Hong Kong, where ethnic Chinese make up a majority of the population (similar to the Singapore context), Hong Kong and Singapore have very different educational systems and cultures (Yang & Lin, 2009).It is thus possible that different results would be obtained across the two settings.The goal of the present study, therefore, was to examine the construct validity of the R-LPQ-2F in a sample of Singapore high school level students.
Based on the 2014 Standards for Educational and Psychological Testing (AERA, APA & NCME, 2014), two types of validity evidence were examined in this study.First, the internal structure of the R-LPQ-2F was evaluated through CFAs which replicated those conducted by Kember et al. (2004) and Phan and Deo (2007).Internal consistencies and inter-correlations between the scales, subscales and components were also examined.Second, relationships between the R-LPQ-2F and two external variables: scores from the Motivation section of the Motivated Strategies for Learning Questionnaire (Pintrich, Smith, Garcia & McKeachie, 1991, 1993) and academic performance, were investigated.Expected patterns of relationships between these variables are discussed in more detail in the Method section.

Participants
Participants were 455 Year 11 students (266 male and 189 female) from a high school in Singapore.The average age of the participants was 16.7 years (SD = 0.39).The participants were enrolled in a two-year physics course taught in the English language, which would prepare them for the General Certificate of Education Advanced Level (GCE 'A' Level) physics examination.The latter examination is administered jointly by the Singapore Ministry of Education and the United Kingdom University of Cambridge International Examinations authority.

Validation Instruments
In addition to the R-LPQ-2F, two further instruments were used in the validation study.First, the Motivation section of the Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich et al., 1991Pintrich et al., , 1993) ) was used.The MSLQ is a well-established instrument in the area of self-regulated learning (Duncan & McKeachie, 2005).Studies of the psychometric properties of the MSLQ have generally produced favourable results.For example, Garcia and Pintrich (1995) demonstrated the sound psychometric properties of the MSLQ using data gathered from 380 students across 14 different subject domains.The Motivation section of the MSLQ includes six scales: Intrinsic Goal Orientation, Extrinsic Goal Orientation, Task Value, Control of Learning Beliefs, Self-efficacy for Learning and Performance, and Test Anxiety.There are a total of 31 item statements, to which students respond on a seven-point scale, ranging from 'not at all true of me' to 'very true of me'.Scores on the R-LPQ-2F were also correlated with physics achievement test scores.The Physics achievement test used in this study was a two-hour pen-and-paper assessment comprising 15 multiple-choice questions and 3 short response questions, adapted directly from past year GCE 'A' level examinations.The test questions, including the marking scheme, were vetted by the subject coordinator and the head of the physics department, who each had more than ten years of teaching experience, to ensure close alignment to the assessment objectives of the GCE 'A' level physics examination (Singapore Examinations and Assessment Board, 2014).The test scores were computed by summing the scores for the multiple-choice questions and the open-ended questions, with a maximum scores of 15 and 65 respectively, giving a total maximum score of 80.As indicated previously, deep learning is generally found to correlate positively with achievement scores, while surface learning has been found to correlate negatively with achievement.The expected pattern of correlations between the R-LPQ-2F and achievement is presented in Table 4.

Procedure
Approval to conduct the research was first obtained from the Human Research Ethics Committee of the authors' research institution.Permission was also granted by the principal of the participating school.At the end of a three-month physics course unit on Newtonian mechanics, all participating students completed the R-LPQ-2F and the motivation section of the MSLQ in a single session.They were provided with hardcopies of the questionnaires and optical mark sheets to shade their responses.Prior to completing the R-LPQ-2F and MSLQ, students were told that their individual responses would not have any bearing on their course grades.In a separate session, they also completed the pen-and-paper closed book physics achievement test.They were given unique codes to use for the questionnaires as well as for the Physics Achievement Test answer scripts, so that their responses could be matched.

Results and Discussion
SPSS version 19 (IBM Corp, 2010) was used to generate all descriptive statistics, Cronbach α coefficients, and bivariate correlations, while LISREL 8.8 (Jöreskog & Sörbom, 2006) was used to perform all CFAs within the study.Prior to conducting these analyses, screening tests for conformity to underlying CFA assumptions were conducted.These tests generally produced satisfactory results.Inspections of z-scores and Mahalanobis distances indicated no univariate or multivariate outliners within the set, at the .001level.There was also no evidence of multicollinearity within the variable set, and no evidence of significant deviations from normality based on skewness or kurtosis coefficients.
Table 5 presents descriptive statistics for items in the R-LPQ-2F.As indicated, the average scores for the Deep Approach scale were higher than for the Surface Approach scale.Interestingly, average scores for Surface Motive were the highest amongst all of the subscales.The latter result was due primarily to the high average scores observed for the two Surface Motive components, Fear of Failure and Aim for Qualification.

Evidence Based on Investigations of Internal Structure
(a) Confirmatory Factor Analyses.CFAs were performed on two first-order models, replicating the analyses performed by Kember et al. (2004) and Phan and Deo (2007).Only first-order models were tested in this study.Three absolute fit indices (the Standardized Root Mean Square Residual, or SRMR; the relative chi-square value, or χ 2 /df; and the Root Mean Square Error of Approximation, or RMSEA) and two relative fit indices (the Comparative Fit Index, or CFI, and the Non-Normed Fit Index, NNFI) were used to assess the fit of each model tested.Accepted cut-offs suggest that in CFAs, good model fit is indicated by a χ 2 /df < 5, SRMR < 0.08, values greater than 0.90 for the CFI and NNFI (e.g., Browne & Cudeck, 1993;Byrne, 1989;MacCallum, Browne & Sugawara, 1996;Schumacker & Lomax, 2004), and RMSEAs of between 0.01 (excellent fit) to 0.08 (adequate fit).The Goodness of Fit Index (GFI) was not used in this study, based on current recommendations within the field (Sharma, Mukherjee, Kumar & Dillon, 2005).Differences between nested models within each of the MSLQ sections were evaluated using the chi square difference test (i.e., Δ χ 2 ).Three sets of models were tested:  Models (a) and (b) were performed with all items within the instrument.Model Relating Ideas, and Understanding. Models (f), (g) and (h) were performed within the Surface Approach scale.Model (f) was a one-factor model for all Surface Approach scale items.Model (g) was a two-factor model, with items grouped into their respective Surface Motive and Surface Strategy subscales.Model (h) was a four-factor, with items grouped into their respective Surface Approach components: Fear of Failure, Aim for Qualification, Minimizing Scope of Study, and Memorization.
The fit indices for the models are presented in Table 6.As indicated, the fit indices for both of the overall scale models (i.e., Models (a) and (b)) approached, but did not meet, suggested criteria.These results contrast with the findings of Kember et al. (2004) and Phan and Deo (2007), who found adequate fit to their data for the overall two-factor model.As indicated in Table 6, Models (c), (d) and (e) for the Deep Approach scale all indicated good fit to the data.Given that the fit indices for the more parsimonious one-factor model were adequate, a one-factor model was deemed most appropriate for the Deep Approach Scale.In contrast, Models (f), (g) and (h) for the Surface Approach Scale indicate that neither the one-nor the two-factor model fit the data well.The four-factor model fit significantly better than the two-factor model, and only this four-factor model meet the fit cut-offs on most criteria (i.e., χ 2 /df, SRMR, RMSEA and CFI).Similar to the results of the Phan and Deo (2007) study, however, the NNFI for the latter model still fell slightly below the accepted cut-off of .90.
Taken together, the most parsimonious model of the R-LPQ-2F that provided adequate data fit in the current study was the one-factor model for the deep learning scale, and the four-factor model for the surface learning scale (Fear of Failure, Aim for Qualification, Minimizing Scope of Study, and Memorization).The final factor models of the Deep Approach and Surface Approach scales are presented in Figures 1 and 2 (respectively).These results suggest that, in the Singapore high school sample, while the Deep Approach scale could be considered unidimensional, the Surface Approach scale could not.(Cronbach, 1970) for the main scales, subscales and components of the R-LPQ-2F are presented in Table 7, together with the findings by Kember et al. (2004) and Phan and Deo (2007).As indicated, the two main scales and most of the subscales achieved sound internal consistencies, and even the components had reasonable coefficients, despite having a small number of items.The α coefficients in the current study were also observed to be generally higher than those in the other two studies conducted by Kember et al. and Phan and Deo.

(c) Intercorrelations within the scale.
Intercorrelations for the R-LPQ-2F scales, subscales and components are presented in Table 8.As indicated, Deep and Surface Approach scores were not significantly correlated, in contrast to the results reported by Kember et al. (2004), who found modest levels of positive correlation between these two broad scales.This finding suggests that for Singapore high school students, deep and surface approaches may not represent two opposing ends of a continuum, as is commonly perceived.It may, therefore, be possible for students to concurrently adopt different learning approaches for a particular learning task.It is also noted that Deep Motive and Deep Strategy scores were highly correlated, as would be expected, but the correlation between Surface Motive and Surface Strategy was only modest.Table 9 shows the intercorrelations between the components of the R-LPQ-2F.Components 1 and 2 relate to Deep Motive; 3 and 4 to Deep Strategy; 5 and 6 to Surface Motive; and 7 and 8 to Surface Strategy.The correlations between the deep learning components (1-4) were positive and relatively high, as expected.With the exception of Aim for Qualification, the surface learning components (5-8) also exhibited similar patterns of correlations, albeit weaker.
Table 9. Intercorrelations between the R-LPQ-2F components In the theoretical framework of the R-LPQ-2F, Aim for Qualification and Fear of Failure, which are classified as surface motives, are perceived to be undesirable because these are likely to lead to the use of surface cognitive strategies, and consequently, to poorer academic outcomes (Biggs, 1993).While the results of this study similarly indicate that Fear of Failure was associated with greater use of surface strategies, Aim for Qualification was not significantly correlated with the Surface Strategy variables.Instead, it correlated positively with two deep motive components (Intrinsic Interest and Commitment to Work) and one deep strategy component (Understanding).These results, therefore, suggest that aspiring for a higher qualification may not be associated with negative outcomes in the Singapore high school context.

Evidence Based on Relationships with External Variables
Correlations between the R-LPQ-2F and scores on the MSLQ Motivation scales, and Physics Achievement Test scores are presented in  Pintrich and Garcia (1991), who found evidence that deep learning can be associated with higher levels of extrinsic motivation.It might also reflect the influence of Confucius Heritage Culture (CHC) of Singapore, which places great emphasis on academic performance and qualifications, as well as the acquisition of deep knowledge (for a more comprehensive review of the CHC, see Watkins & Biggs, 1996).In contrast, the patterns of correlation obtained for Surface Motive and the two components of Surface Motive (Fear of Failure and Aim for Qualification) did not align well with expectations.While all three variables did correlate positively with the two negative MSLQ variables, amongst the three, only Fear of Failure was negatively correlated with physics achievement.Fear of Failure was also, as expected, negatively correlated with Self-Efficacy for Learning and Performance.Surprisingly, however, both Surface Motive and Aim for Qualification were positively correlated with two of the positive MSLQ motivation scales (Task Value and Control of Learning Beliefs).Aim for Qualification was also positively correlated with Self-Efficacy for Learning and Performance.Thus, of the Surface components, the one that exhibited the most unexpected pattern was Aim for Qualification.

Conclusions and Implications
Results of this study indicate that the internal structure of the R-LPQ-2F in Singapore secondary students is similar to the original factor structure proposed by Kember et al. (2004).In this study, a structure comprising one deep factor and four surface factors fit the data well.Cronbach's α coefficients for the scales, subscales and components were generally also acceptable.Results of the intercorrelations within the R-LPQ-2F also supported the validity of the instrument.As the R-LPQ-2F is readily available and relatively brief, it provides an attractive option for educators to use in their ongoing practices.The results of this study affirm that the instrument is suitable for assessing students' learning approaches within Singapore classrooms.
Results of the correlations between the R-LPQ-2F Deep Approach scale and for the Surface Strategy subscale, MSLQ motivation scores, and physics achievement scores, similarly aligned with expectations derived from the theoretical basis of the R-LPQ-2F.This was not so, however, for the relationships between the Surface Motive subscale, MSLQ motivation, and physics achievement scores.This can be attributed primarily to the contribution of the Aim for Qualification component of the Surface Motive subscale.While these results were not expected based on previous results reported from Western countries, they do align well with emerging views of the surface and deep motivations held by many Asian students.For example, Boekaerts (2003) argued that, in contrast to Western cultures, factors such as personal ambition and material reward will prompt many Chinese students to adopt deep learning strategies.Ramburuth and McCormick (2001) similarly reported that Asian students will often combine surface strategies with intrinsic or deep motivations in approaching their learning tasks.Based on these results, assumptions made about the negative implications of motivations classed as 'surface' oriented in Western countries may not hold in Asian samples.
Future research is needed to investigate implications of the Surface Motive components of the instrument within the Singapore context.It should be noted here also that this study only focused on upper secondary level students.Thus, it is not known based on these results whether the instrument can be used in other levels of the Singapore system.Given its potential utility in this context, further work could also be conducted to determine the suitability of the R-LPQ-2F in earlier grade levels.
Overall, the results of this study provide strong support for the R-LPQ-2F to assess students' learning approaches in Singapore secondary schools.While many teachers may already be adopting strategies to encourage the use of deep learning approaches in their classrooms, it is possible that they are not effectively monitoring the progress of their students in this domain.The R-LPQ-2F can be a very useful tool for such a purpose.This instrument has various other advantages, which make it highly suitable for use in classroom contexts, which include the fact that it is readily downloadable online, free to use, and, with only 22 items, efficient to administer and score.Collectively, these features suggest that the R-LPQ-2F has considerable potential as a tool to monitor students' learning processes, with the ultimate goal of enhancing overall schooling outcomes.

(
21) I like to do enough work on a topic so that I can form my own conclusions before I am satisfied.I try to relate what I have learned in one subject to what I learn in other subjects.(6) I like constructing theories to fit odd things together.Understanding (10) I try to relate new material, as I am reading it, to what I already know on that topic.(14) When I read a textbook, I try to understand what the author means.
(a) was a two-factor model, in which all items were grouped into their respective Surface and Deep Approach scales.Model (b) grouped items into the four subscales within the instrument (i.e., Surface Motive, Surface Strategy, Deep Motive, and Deep Strategy). Models (c), (d) and (e) were performed within the Deep Approach scale.Model (c) was a one-factor model for all Deep Approach scale items.Model (d) was a two-factor, with items grouped into their respective Deep Motive and Deep Strategy subscales.Model (e) was a four-factor model, with items grouped into their respective Deep Approach components: Intrinsic Interest, Commitment to Work,

Figure 1 .
Figure 1.Final one-factor model for the Deep Approach Scale

Table 1 .
Structure and items in the R-LPQ-2F

Table 1 .
Structure and items in the R-LPQ-2F (Continued)

Latent Factors Fit Indices Kember et al. (2004) Phan & Deo (2007)
analysis using a sample of 2295 students from 35 high schools in the Fiji islands.Results for this study are also summarized in Table2.As indicated, the fit indices of the two-factor model (i.e., Model I: Deep Approach and Surface Approach) were again adequate, though the NNFI was somewhat low.The separate CFAs conducted for the four components of the , the αs for all four of the subscales (i.e., Deep Motive, Deep Strategy, Surface Motive, and Surface Strategy) fell below 0.60.Internal consistencies were not reported for the eight individual subscale components in this study.

Table 3
Curran & Bowie, 1998)ed pattern of correlations between the R-LPQ-2F scales, subscales and components and the MSLQ motivation scales.Variables relating to deep learning are generally expected to show positive correlations with the 'positive' motivation constructs measured by the MSLQ (i.e., Intrinsic Goal Orientation, Control of Learning Beliefs and Self-efficacy for Learning and Performance) (e.g.,Biggs & Tang, 2011).Conversely, variables related to surface learning are expected to show negative correlations with the 'positive' motivation constructs in the MSLQ, and positive correlations with the 'negative' motivation constructs in the MSLQ (e.g.,Curran & Bowie, 1998).

Table 3 .
Expected pattern of correlations between the R-LPQ-2F, MSLQ, and achievement

Table 4 .
Descriptive Statistics for main scale, subscale and components of R-LPQ-2F

Table 10 .
As expected, the Deep Approach scale, subscale, and component scores all correlated positively with the 'positive' MSLQ motivation scale scores.With the exception of two components (Commitment to Work and Relating Ideas), all the Deep Approach variables also correlated positively with physics achievement.Surprisingly, the MSLQ Extrinsic Goal Orientation scale correlated positively with all of the Deep Approach variables, and similarly, the MSLQ Test Anxiety scores also correlated positively with the Deep Approach scale score, as well as with Deep Motive and Commitment to Work scores.The latter two sets of results are surprising, given that both Extrinsic Goal Orientation and Test Anxiety are generally perceived to represent 'negative' aspects of motivation.The results are, however, aligned with results reported by

Table 10 .
Correlations of R-LPQ-2F with MSLQ motivation variables and Physics Achievement Test scoresResults for the overall Surface Approach scale and the Surface Strategy subscale and component scores were also generally aligned with expectations.Surface Approach scale scores correlated negatively with three of the four positive MSLQ motivation variables (Intrinsic Goal Orientation, Task Value, and Self-Efficacy for Learning and Performance) and with physics achievement, but positively with the two negative MSLQ motivation variables (Extrinsic Goal Orientation and Test Anxiety).Surface strategy subscale scores and the two components of Surface Strategy, Minimising Scope of Study and Memorization, correlated negatively with all positive MSLQ variables and with physics achievement, while correlating negatively with the two positive MSLQ motivation variables of Extrinsic Goal orientation and Test Anxiety.