Transfer of Learning for Evidence-Based Practice in Psychology

The aim of this paper is to assess the transfer of learning of students from a master’s degree in psychology through observation of their skills in information assessment and intervention. The participants were 10 incoming students of a professional-aimed master’s program in psychology offered in a Mexican public university. They carried out two types of tasks: a) review of a scientific text and b) interaction in intervention with clients, where the participants were evaluated by observational strategies. The results reflect zero relation between the level of skill displayed by most students in their information assessment skills and their intervention skills, which could suggest a low probability to conduct evidence-based practice. The data presented provide information for the design of graduate programs in order to foster the transfer of learning of these skills, which is essential for evidence-based practice.


Introduction
The psychologist is an agent of change within the institutions and society in general. Therefore, the study and analysis of their educational process is of great relevance, since the quality and effectiveness of the psychologist's actions for the solution of social and scientific problems depend on said process (Morales, 2012;Santoyo, 2012).
The National Center for the Evaluation of Higher Education ('Centro Nacional para la Evaluación de la Educación Superior ', CENEVAL, 2018) reported that in Mexico during 2017, a total of 11,309 students took the General Undergraduate Examination (Examen General de Egreso) of the Bachelor's degree in Psychology. This examination evaluates three areas: a) evaluation, b) intervention and c) measuring and research. Based on the score obtained, the examinees could get three possible results: 1) unsatisfactory performance, 2) satisfactory performance and 3) outstanding performance. The overall results of the examination showed that 51.5% of the examinees obtained unsatisfactory performance, 43.8% satisfactory performance, and only 4.7% achieved outstanding performance. When analyzing results per area, the trend remains similar to the overall one. These results are evidence of the educational needs of psychologists in areas that are essential for making a difference in institutions and the society. These areas are critical components for accomplishing a quality professional practice.
Education in psychology faces a number of challenges, one of them is to encourage those who are in this process to get interested in the new research findings developed in the profession and that they implement those, hence generating bridge research. Translational research refers to a dynamic process that encompass from the discovery of new knowledge to its adoption and use in natural scenarios of everyday life in order to generate the social welfare pursued (Ramis et al., 2019;Santoyo, 2012). To accomplish the latter, professionals are required to have basic and technical knowledge specific to the discipline, subsequently professionals are assessed and transferred for their application in the solution of problems of social relevance (Paravattil, Shabana, Rainkie, & Wilby, 2019).
In this sense, the situation of the translational research with the Evidence-based Practice (EBP) is of the utmost importance. The EBP is defined as the decision-making process involved in the solution of scientific and social problems that must be carried out based on the best available evidence (American Psychology Association [APA], 2006;Ybarra, Orozco, & Valencia, 2015). This concept has been adopted from the Evidence-based ies.ccsenet.org International Education Studies Vol. 13, No. 5; Medicine and it has recently been studied above all else in areas such as nursing, physical rehabilitation (Kilicli, Kelber, Akyar, & Litwack, 2019;Saunders et al., 2019) and less frequently in psychology, due to the fact that research in this discipline has focused on the dissemination and treatment implementation, rather than related to learning skills involved in the PBE (Pedroza & Martínez, 2012) It is on the last-mentioned area where the elements of implementation and dissemination of treatments are emphasized, more than that related to learning skills involved in the EBP.
For the training of students in the EBP, several strategies have been employed. These are referred to as passive strategies or active learning strategies. The strategies of passive learning entail the reviewing of texts and manuals that provide information about the treatments. These strategies require the identification of scientific information and the assessment of the quality of scientific, peer or personal evidence (Rousseau & Gunia, 2016).
The active learning strategies include behavioral essays, feedback, and role-playing. These strategies are directly linked to the implementation of interventions, which requires that the student displays skills for intervention and evaluation of the results obtained from the application of said interventions (Jiménez & Santoyo, 2016;Paravattil, Shabana, Rainkie, & Wilby, 2019).
A critical ability for the EBP consists in the search and assessment of bibliographic information. It has been documented that the learning of this skill in professional education is linked to a greater likelihood of performing EBP once the student concludes their studies (Fresneda, Muñoz, Mendoza, & Carballo, 2012).
However, professionals rarely do research in scientific texts before designing or implementing a treatment. Vargas et al. (2016) assessed the reading habits of 102 health professionals of a primary health care center for addictions during the professional education and their professional practice. The results were as follows: while studying, 23.53% of them read scientific articles and subsequently applied the findings in their practice, whereas by questioning them about their usual clinical practice, 28% referred to read and apply the findings at work.
Nevertheless, the information search is not enough to assess the available evidence. In this respect, the lack of skills to assess scientific evidence has been reported as a barrier for the EBP (Williams & Beidas, 2019). The ability to identify, to assess and distinguishing between reliable sources from those who are not, is linked to better results in the implementation process (IIlic, 2009). The search of valid information, its critical reading and the incorporation of the knowledge that is gradually acquired along with the professional's practice are tasks that constitute the continuous education of the clinical specialist. It must be emphasized in the relation between science and profession (Aglen, 2016). Espinosa, Santoyo, and Colmenares (2010), carried out a study in which they assessed and taught the methodological and conceptual skills for the analysis of scientific texts in psychology, finding that the level of performance of bachelor's degree's students increased from 50% to 90%. The students had a greater difficulty in performing tasks that required information analysis. Even though the model from which this study arises assumes that these abilities are linked to intervention skills, this relation has not been documented.
Furthermore, the aforementioned active learning strategies for the EBP are designed to enable professionals to learn intervention skills. This has led to the EBP concept to be associated only with terms such as treatment fidelity or quality of implementation, leaving aside the entire decision-making process: from evidence identification to the selection of the best and their correct implementation (Stains & Vickrey, 2017;Wiltsey et al., 2015).
However, studies regarding learning for EBP have considered each of the skills involved in the process separately. Therefore, it is important to note the relevance of transfer of learning of skills for the assessment of scientific information to the implementation of interventions based on such evidence.
The aim of this paper is to evaluate the skills for information assessment and for implementation of a group of master's students to establish possible links between them

Participants
This study was conducted with 10 incoming students of a professional-aimed master's programme in a public university. Three of them were men and the rest women; the average age was 26 years old. It is worth noting that in average it had been two years since they had graduated from undergraduate degrees.

Instruments and Materials
1) Answer sheet for the strategic text analysis (Jiménez, 2016 2) Rubric for the evaluation of the text analysis task (Jiménez, 2016). Contains each of the categories of the MAE txt (Santoyo, 2001), which can be evaluated on a scale of 0 to 4 points. The reliability of the data derived from the instrument is given by the level of agreement between evaluators, which must be over 80%.
3) Behavioral observation system of intervention skills (Jiménez & Santoyo, 2016). It consists of 22 behavioral categories, organized in three groups: a) behaviors that can be emitted by the client and by the therapist (student), b) categories that can only be emitted by the student in their role as a therapist and c) behaviors that can only be emitted by the client. The quality of the data in this type of instrument is given by 80% agreement among observers (Bakeman & Gottman, 1989). In this study, the interobservers agreement was of 89%.

Design
It consisted of an observational design of an ideographic, punctual and multidimensional type (Anguera, Blanco, Hernández, & Losada, 2011), therefore the individual analysis of the participants during their postgraduate admission is considered, and skills for the PBE in two dimensions are analyzed: information evaluation skills and intervention skills.

Procedure
The participation of the 12 incoming students of a professional-aimed master's degree was requested, 10 of them gave their consent to collaborate in the study.
The data were collected during the beginning of the first semester of the students' master's degree. First, the collection of data of information assessment skills was carried out in a classroom of a psychology faculty of a public university. For this purpose, each participant was given a 4-lenght classic article about behavior modification written in Spanish along with the answer sheet for the strategic text analysis (Jiménez, 2016). The written instructions for the task were attached to said sheet; students were instructed to read the whole text and then answer the questionnaire. They could highlight the article or write notes while reading, and there was no time limit to complete the analysis. Each questionnaire was evaluated based on the rubric mentioned in the instrument section.
At the end of the text analysis task, students were cited for the following week, in which the observation of a simulated situation of interaction with a patient would be carried out. Prior to the observation session, two undergraduate psychology students were trained to act as confederates (patients). The training requested the confederate to emit resistant behaviors with intervals of approximately one minute; consequently, the resistant behaviors were identical in topography and they occurred regardless of the behavior of the evaluated subject. In this way, the conditions were created to observe the skills with which the aspirants handled the resistant behaviors of the confederates. Moreover, in this manner, it was ensured that all participants were evaluated under equal conditions.
The simulated situations were recorded and were conducted in a psychological care center where participants regularly go. The duration of the session where each participant was observed was of 20 minutes.
Once the videos of the interaction in an intervention situation were obtained, observation records of partial interval of five seconds were conducted. This made possible to obtain the probability of occurrence of the students' behavior when interacting with the confederate (patient).

Results
The skills of the incoming students in the text analysis task were analyzed. The categories of the MAE txt were grouped in five types of skills: identification, comprehension, analysis, deduction, and evaluation. The average score for each type of skill was obtained. In Figure 1, it can be observed that when considering the average performance of the participants, they obtain scores close to 1.5 in the tasks of identification, comprehension, and information analysis (SE=.26, .21 and .10 respectively), whereas in categories involving deduction and evaluation, the average performance is 2 (SE=.19) and 2.1 (SE=.15).  Vol. 13, No. 5; On the other hand, in the same table, it can be noted that participant number 4, gets a .7 probability for the handling of resistant behaviors in an appropriate manner; nevertheless, the scores that this participant gets in the MAE txt task correspond to incorrect answers in all categories. Note. P1, P2…P10 indicate the participant to whom the data of each row are referred. The participants in bold are those with the best performance.

Discussion and Conclusion.
The evidence-based practice has been limited to the study of the full application of scientifically validated treatments, leaving aside the decision-making process that leads to the choice and implementation of a treatment. Hence, the importance of this study, whose results indicate the educational needs of future psychologists, in pursuit of their interventions generating effective results for society.
This paper originated from the hypothesis regarding a link between information assessment skills and intervention skills. The findings of this study show that of the five skills described (comprehension, identification, analysis, deduction, and evaluation) in the MAE txt (Santoyo, 2001) is in the information analysis skill where participants show the lowest performance. These data are congruent with what was reported in previous studies with undergraduate students (Espinosa, Santoyo, & Colmenares, 2010;Jiménez, 2016). These data are worrisome if we consider that the participants of this study are graduates on psychology and with at least two years of professional experience and one of the goals of learning at the undergraduate level is to interpret psychological phenomena based on scientific reasoning to generate research and intervention plans (APA, 2013), which requires the ability to analyze models and explanations, and not just describe them.
The students evaluated have low performance in the MAE txt task; however, those with better scores, obtained greater conditioned probabilities of handling clients' resistant behaviors through desirable behaviors, in intervention situation, except in the case of participant number 4. This participant obtained all scores below expectations, and yet resolves the client's resistant behaviors with the same conditioned probability as participant number 6. When analyzing the data globally, no correlations were found between both types of skills. This could suggest that while therapists are aware of a protocol of intervention and they executed it correctly, to leave aside the basis of it can hinder its adaptation and subsequent adoption, thus altering the results in terms of the effectiveness of the intervention (Stirman et al., 2015). Therefore, it is important to conduct more studies in which both types of skills are analyzed and to consider the effectiveness of the interventions of therapists, taking into account the change that arises in the behavior of their clients.
Furthermore, these same results suggest that decision making in psychological interventions is made on the basis of common sense or intuitions, which generates costs to institutions that receive their services given the questionable efficacy of treatments (Klenowski, Bell, & Dodson, 2010); besides, it has an impact on the population requiring effective solutions.
In addition, specific variables involved in decision making such as clinical experience and the theoretical approach assumed in implementation should be analyzed, which could mediate the approach to evaluation of ies.ccsenet.org International Vol. 13, No. 5; scientific knowledge and the application of interventions (Brookman-Frazee, Haine, Baker-Ericzén, Zoffness, & Garland, 2010). That would provide information about elements to strengthen in the theoretical and practical training of psychologists.
In this respect, Rousseau and Gunia (2016), also propose that the ability, motivation and opportunity to the practice could help to understand the PBE, so it is necessary to evaluate in studies later, the technological skills involved in the psychological implementation; coupled with considering the motivational factor, which is related to reinforcement that the professional in training obtains for the execution of the previous skills, because it is possible that if the professional believes that making inconsistent modifications to the programs is equally effective than applying them with fidelity, it maintains this kind of intuitive practices. Finally, the other variable of the model is the opportunity for evidence-based practice, which makes it necessary to analyze the elements of the context that enable PBE to occur.
In prospective, it is necessary to carry out an analysis of the psychology curricula to identify whether in the objectives or subjects' syllabus, analysis, deduction, evaluation and comprehension are actually being fostered. And on the same note, it also becomes important to identify how the theoretical content is linked to the application in order to determine the skills that are being fostered for decision-making based on scientific evidence.
In addition, it is suggested to review the profile of students entering psychology programs, because according to the National Association of Universities and Institutions of Higher Education (ANUIES) in 2012-2013, there were 147,093 students enrolled in psychology programs in Mexico; this situation implies a challenge for teachers who seek to foster skills among their students to solve social and scientific problems.
The above should be considered when analyzing the opportunities for teaching and learning of the PBE; especially when considering the use of passive learning strategies (Kaplan, Rossenfeld, & Haber, 2003;Vargas et al., 2016) and active (Beidas & Kendall, 2010) at different times of education, which require human and institutional resources differentials, as they constitute the context of the training of psychologists in Mexico.