Students’ Task Understanding during Engineering Problem Solving in an Introductory Thermodynamics Course

Understanding problems or tasks is a critical step in any problem-solving activity and the heart of self-regulated learning. When encountering a problem, students draw upon information available in the environment, along with knowledge, concepts, and perceptions derived from prior learning experiences, to interpret the demands of the task. Interpretation of tasks is, therefore, a key determinant of the goals set while learning, strategies selected to achieve those goals, and the criteria used to self-assess and evaluate outcomes. The purpose of this study is to better understand engineering students’ self-regulation in task interpretation processes while engaged in problem solving in an introductory engineering thermodynamics course. Two research questions guided the study: (1) What are the gaps, if any, between the instructor’s and students’ interpretation (explicit and implicit task features) of a problem-solving task?; and (2) How do students’ task interpretation (explicit and implicit) change after engaging in self-evaluation of their problem-solving processes? One hundred twelve (112) second year engineering undergraduates voluntarily participated in the study. Analysis of the data collected revealed a significant difference between the instructor’s and students’ task interpretation of the assigned problems. Furthermore, the analysis showed that students’ had a higher ability to identify the explicit parts of problem tasks than implicit ones. Students were able to grasp 63 to 77 percent and 39 to 49 percent, respectively, of the explicit and implicit information that was presented to them while engaged in problem-solving activities.


Introduction
Understanding the problem is a critical step in any problem-solving activity.Student's understanding of an assigned task determines the strategies they select for completing the task (Butler, 1998).Task interpretation (TI), which refers to how students perceive an academic task (Butler, 1998), has emerged in the literature as an important component of self-regulated learning (SRL) and, consequently, academic success (Andrade & Valtcheva, 2009;Boekaerts, 1997;Coutinho, 2007;Downing, Kwong, Chan, Lam, & Downing, 2009;Ferla, Valcke, & Schuyten, 2009;National Research Council, 2000;Otero, Campanario, & Hopkins, 1992).TI is the first step in the self-regulated learning (SRL) process because students' interpretation of the task assigned is a key determinant of the goals they set while learning, the strategies they select to accomplish the goals, and the criteria they use to self-evaluate outcomes for academic success (Butler & Cartier, 2005).
Engineering problem solving requires students to apply scientific and mathematical principles to develop quantitative estimates of physical phenomena (Engineering Accreditation Commission, 2003).Similarly, learning to solve problems effectively and confidently often requires students to engage in problem-solving practice and critical self-reflection over a sustained period of time.The focus of this study was students' task interpretation during problem-solving activities in an undergraduate engineering thermodynamics course.This course purposefully emphasizes the combination of problem-solving activities and individual self-evaluation of problem solutions in order to promote conceptual understanding through application and reinforcement.Understanding a problem is the first critical step leading to strategies to achieve the solution.Problem definition is always the first step in any model for problem-solving process (Jonassen, 2004;Kim & Kim, 2015;Koichu & Leron, 2015;Polya, 1957).However, successful problem definition requires a strategy to understand the problem.
As a cognitive control process, SRL is tied to metacognition.Emphasizing the use of the terms 'metacognition' and 'self-regulation', Dinsmore, Alexander, and Loughin (2008) stated that, "We see a clear cognitive orientation of metacognition, while self-regulation is as much concerned with human action than the thinking that engendered it" (p.405).Furthermore, researchers have maintained that the important issue in self-regulation and metacognition is to understand "the correspondence between metacognition and action.How do thoughts and feelings of learners guide their thinking, effort, and behavior?"(p.21) (Paris & Winograd, 1990).
The dynamic and iterative interplay between metacognitive and cognitive activities is described by Butler and Cartier (Brydges & Butler, 2012;Butler & Cartier, 2004, 2005;Butler, Schnellert, & MacNeil, 2015;Butler & Winne, 1995;Cartier & Butler, 2004) in their SRL model, which characterizes SRL as a complex, dynamic, and situated learning process (see Figure 1).This model consists of a learner, a learning environment, and learner's continuous engagement with the confined environment in various contexts (e.g., instructional approach, discipline, or faculty).The learner refers to one's experience, strengths, challenges, metacognition skills, knowledge, and beliefs.The learning environment refers to the given task or activity, available resources, and feedback.The learning engagement refers to continuous activities of understanding a task, developing a plan according to task understanding, acting on the plan developed, monitoring progress and results, and adjusting approach if necessary.Further, the model also proposes that learning engagement is affected by learner's emotions and motivations.Hadwin, Oshige, Miller, and Wild (2009) offers a complementary model that further describes the explicit, implicit, and socio-contextual aspects of task interpretation.In this study, explicit features of assigned tasks include information that is overtly presented in problem descriptions, such as the task goal or requirements.Implicit features of a task include any information beyond the problem description, such as relevant concepts or procedures.Socio-contextual aspect refers to learner's awareness about the contexts surrounding the task, which is considered as part of the layers of contexts in the Butler and Cartier's model.Mairing (2016) reported that learners' conceptual knowledge and structure influence their task interpretation, such that a naïve problem-solver tends to have a limited understanding of a problem; these results are consistent with the models described in this study (Brydges & Butler, 2012;Butler & Cartier, 2004, 2005;Butler et al., 2015;Butler & Winne, 1995;Cartier & Butler, 2004;Hadwin et al., 2009).Lubis and Nasution (2017) argue that each learner's initial task interpretation is unique because they have different levels of conceptual understanding.Fortunately, students' understanding has been shown to evolve as they work through the problem (Rivera-Reyes, 2015; Rivera-Reyes, Lawanto, & Pate, 2017) and interact with their peers (Rivera-Reyes, Lawanto, & Pate, 2016).As their task interpretation mature, students also adjust their approach to solving the problem (Abdillah, Nusantara, Subanj, Susanto, & Abadyo, 2016;Kahneman, 2003).Moore, Miller, Lesh, Stohlmann, and Kim (2013) argue that due to the nature of engineering problems, there are some constraints that cannot be easily identified from the beginning, which then require the engineers to work iteratively through the problems.

Purpos
The purpo interpretat course.Qualitative data were collected through Task Analyzer Questionnaire (TAQ) to assess the participants' task interpretation.Unique TAQ was developed for each specific assignment (i.e., the three problem solving assignments described earlier).The TAQ has been face-validated, construct-validated, and pilot-tested, and then revised accordingly prior this study.Each TAQ consisted of eight open-ended questions and included items related to both explicit and implicit aspects of task interpretation as described by Hadwin et al. (2009).Examples of an explicit and implicit task interpretation questions were "What were your goals in solving this problem?(In other words, what were you asked to do?)" and "List the major concepts and/or principles discussed in class that you used in solving this problem", respectively.
Two raters assessed student's task interpretation of each problem: the course instructor and another engineering faculty member who was also a content expert.In order to generate the TAQ responses used to rate student responses, the instructor provided initial TAQ responses.Next, the content expert evaluated these responses.After discussions between the instructor and content expert, revisions were made to the instructor's responses.The revised responses were used to score students' answer.
Students' TAQ scores ranged between from 0 to 2. A TI score of 0 was assigned to a blank or incorrect answer; a score of 2 was given to a correct answer (i.e., when students were able to describe at least half of the possible correct responses); and a score of 1 indicated an incomplete answer.The score applied to an incomplete answer was mutually agreed on by the raters after comparing the incomplete answer to the problem solution.Together, the two raters achieved an inter-rater reliability score of 97 percent agreement.
To analyze the score, a comparison of the means of students' TAQ responses before and after self-evaluation of the problem solution was conducted.Next, two-tailed paired-sample t-tests were conducted.A cutoff value of .05 for Type 1 error was used to determine whether the results of the TAQ before and after are significant.

Addressing Research Question 1: What Are the Gaps, If Any, between the Instructor's and Students' Interpretation (Explicit and Implicit Task Features) of a Problem-Solving Task?
To investigate whether there were any gaps between the instructor's and students' interpretation of a task, three sets of paired-t-tests were conducted (see Table 1).The first test set was conducted to evaluate the mean differences between overall (i.e., combined explicit and implicit) task interpretation scores of instructor and students (i.e., overall students' TI before and after engaging in self-evaluation of the solution of a problem solving task).The results suggest that: (1) There was a significant difference of overall TI score between the instructor and students before engaging in self-evaluation of the solution of a problem solving task: t(103) = 35.409,p < .001;and (2) There was a significant difference of the instructor's and students' overall task interpretation scores after engaging in self-evaluation of the solution of a problem solving task: t(111) = 37.42, p < .001.Moreover, it was also found that students' overall TI score before engaging in self-evaluation of the problem solution was 0.998 (i.e., 50 percent) and after engaging in self-evaluation of the problem solution students' overall TI score increased to 1.110 (i.e., 56 percent).Although there were significant differences between instructor and students' overall task interpretation scores, the analysis showed that students' self-evaluation of the problem solution activity did improve by 6% student's understanding of the problem.
The second and third sets of paired t-tests evaluated the mean differences between instructor's and students' explicit and implicit task interpretation scores before and after engaging in self-evaluation of the problem solution.The findings suggest that: (1) there was a significant difference between instructor's and students' explicit task interpretation scores before and after engaging in self-evaluation of the problem solution; and (2) there was a significant difference between instructor's and students' implicit task interpretation scores before and after engaging in self-evaluation of the problem solution, see Table 1.For the first problem (i.e., Task 1 -the least difficult problem), the findings suggest that there was a significant improvement of students' overall task interpretation score after engaging in self-evaluation of the solution of a problem-solving task, t(85) = 2.50, p < .05.However, when evaluating the explicit feature of the students' task interpretation, the findings showed no significant improvement of students' explicit task interpretation score, t(85) = 1.19, p > .05.A significant improvement was found on the implicit feature of students' task interpretation, t(85) = 2.75, p <.01, see Table 2.This suggests that although students' explicit TI score was slightly higher after engaging in self-evaluation of the problem solution, it seemed like student self-evaluation of the problem solution did improve significantly their implicit task interpretation score.For the moderately difficult problem (i.e., Task 2), no significant change of students' overall task interpretation score was found, t(97) = .706,p > .05.Similarly, no significant change was also found for the explicit and implicit features of students' task interpretation scores, t(97) = .618,p > .05 and t(97) = .316,p > .05,respectively, see Table 3.However, when comparing students' TI explicit and implicit scores before and after students' self-evaluation of the problem solution, across these three problems, the findings suggest that students' TI implicit scores were significantly lower than their paired TI explicit scores, see Tables 5 and 6.These lower TI implicit scores compared to its explicit scores were consistent with the previous findings.

Conclusions and Discussion
It was not a surprise to see that there was a gap between what relevant information (explicit and implicit) of a problem is essential for the instructor and students.The analysis of the t-test revealed a significant difference between the instructor's and students' task interpretation scores of all assigned problems (as shown in Table 1).One possible explanation for this phenomenon is that experts' cumulative problem-solving experiences enable them to develop complex schema structure which sometimes make them oblivious to the complexity of their thinking process (Jones & Idol, 2013).
Further analyses of the t-tests revealed a significant difference between the students' ability to identify the explicit and implicit information associated with the assigned task (see Tables 2-5).Students' scores indicate a higher ability to identify the explicit understanding of problem than the implicit one.This suggests that the students seem to experience more challenges to identify information beyond the problem description, such as the purpose for the problem assigned, connections to learning concepts, and potential resources for completing the task.While implicit information often seems obvious to the instructors, students may face difficulty in making the connections between information that is presented in problem description (and class discussion) and information that the students need to extrapolate beyond the problem description.Furthermore, problem complexity might influence students' ability to extrapolate relevant information from the problem (as seen on the lower Implicit TI score of Task 3).Van Meter et al. (2016) argue that students' conceptual understanding and reasoning can be improved through self-regulation practices (i.e., organizing, elaborating, and monitoring).
Despite higher explicit task interpretation scores, instructors may not expect their students to be able to grasp all of the explicit information given by the problem description or discussed during class.The findings show students' explicit task interpretation scores ranges from 1.257 to 1.531 (on a 0-2 scale).It appears that students in this study were able to grasp 63 to 77 percent of information explicitly presented to them while engaged in problem-solving activities.This finding aligns with the novice-expert research findings that suggest experts spend more time on understanding the task and engaging in monitoring and evaluation, in the attempt to develop more complete representation of the problem, before finding the appropriate strategies to solve problems (Abelson, 1981;Glaser, 1992;Herbig & Glöckner, 2009;Hoffman, 1996;Lesgold et al., 1988;Soloway, Adelson, & Ehrlich, 1988).
Although it is inconclusive, students' self-evaluation of the problem solution may help improve their task interpretation skills (see Tables 2-4); however, this study shows significant improvement only on students' implicit task interpretation skills while solving the least difficult problem.Self-evaluation of the problem solution did not seem to significantly improve students' task interpretation scores when it comes to solving problems that are more difficult.These non-significant TI changes may be caused by students' ability to learn the TI strategy from self-evaluation of the problem solution on the first problem (i.e., the least difficult problem), or students may have difficulty identifying essential implicit information necessary (e.g., relevant concepts) to solve problems that are more complex.From an instructional perspective, it is also possible that students did not fully utilize self-evaluation due to lack of more specific instruction.Numerous studies argue that a strong instructional guidance is more suitable for novice and intermediate learners (Ackermann, 1996;Kirschner, Sweller, & Clark, 2006;Reder, Anderson, & Simon, 1996).Further investigation is needed to better understand how students' self-evaluation of the problem solutions can be used to improve both explicit and implicit task interpretation, as well as other SRL features, for problems of varying complexities.

Table 1 .
Gaps between instructor's and students' task interpretation of all assigned problemsBefore self-evaluation of the problem solution After self-evaluation of the problem solution

Table 5 .
Students' explicit and implicit task interpretation scores before self-evaluation of the solution across 3 problems