What Works ? The Culture of Evidence in University Teaching

This article analyses the culture of evidence in university teaching and its implications in the professional training of teachers in higher education. The new culture of organisation and assessment introduced into university teaching has brought about the configuration of a management model geared towards results and accountability based on solid evidence. Its implementation means that both administrators and teachers are asking themselves: what works? This study shows that the implementation of a culture of evidence requires the adoption of a pluralist vision of evidence, as well as clear criteria for determining the validity of evidence. In addition, teachers should be trained to mobilise systematic pedagogic knowledge and transform their practice, using available institutional support, the systematic analysis of their own experience, and the promotion of best practice.


Introduction
Philosophy, starting with Socrates, implemented questioning as a strategy for thinking.While the culture of questioning has benefited philosophy, science, and technoscience, the nature of questions varies depending on the logic of the system of reference.For example, fact-based questions, such as What is the predominance of a specific phenomenon or problem? are used to generate knowledge in scientific research, whereas questions such as What is the impact of a specific action in the improvement of a specific problem?are geared towards improving the action implemented, ensuring accountability, and generating accurate information for the implementation of future possible actions.
The title of this paper starts with a question: what works?This question-increasingly present in public sector institutions-has always characterised the sphere of public policies.In education this question is often prompted by the internal reforms being implemented, as well as by the new organisational culture based on the Total Quality approach.This approach, which attempts to institutionalise assessment culture, reinforces the referential function with more complex assessment processes and systems.
It is hardly surprising that the 21 st century culture of assessment, based on solid evidence, accountability, and the logic of technoscience (knowledge to transform) has emerged mostly in the fields of education, medicine and administration.On an international scale, in 2003 the Centre for Educational Research and Innovation (CERI) of the Organisation for Economic Co-operation and Development (OECD) promoted a new work agenda focused on research geared towards political decision-making: Evidence-based Policy Research in Education (EBPRE).Its aim was to encourage international research in order to facilitate political decision-making on innovation in education, but its scope gradually widened from teaching-learning contexts in the school system to those in the university system.
For instance, the 2012 OECD report Fostering Quality Teaching in Higher Education (Hénard & Roseveare, 2012) states the need to promote evidence-based self-assessment processes for the innovations implemented by teachers, while also encouraging the promotion and exchange of best practices in university teaching.
International bodies unanimously promote the culture of evidence in university education as a method for encouraging innovative and flexible educational practices.Thus, in addition to being asked to provide contrasted information on the learning attained by their students, university teachers are often encouraged to take part in evidence-based professional development and pedagogic innovation programmes in order to improve the quality of university teaching.This can clearly be seen in the work carried out by the High Level Group (HLG) on the modernisation of higher education (HLG, 2013), stressing the importance of enabling teachers to design personalised evidence-based learning settings.This paper holds that beyond the question what works?there underlies a new culture of organisation and assessment in universities with implications for the professional training of teaching staff.This requires the promotion of a scientific and professionalising pedagogic culture among teachers through systematic and empirical analysis of teaching, as well as the promotion and exchange of best practices, incorporating proven teaching strategies and educational techniques.
This study is structured on five main points.Firstly, the evidence-based approach (EBA) is examined in the context of the current challenges in university teaching.This is followed by a discussion on the possibilities and limitations of the EBA in improving the quality of teaching, before going on to examine the reasons which drive the research-based teaching from the new assessment and pedagogic culture now expected from university teachers.The fourth section of this paper analyses how this assessment experience is to provide new information, before finally examining the process of transferring evidence to teaching and integrating it into practice.

The Evidence-Based Approach and the Challenges of University Teaching
Universities are immersed in deep and complex challenges as a result of globalisation, the advance of science and technology, and the paradigmatic change in the production system and in social relationships.This in turn means that they are forced to deal with the challenges of the Society of Knowledge and the new Digital Economy, deeply transforming their organisation structure and teaching practices.The new demands of society, especially the production sector and students, are forcing universities to refocus their goals, further committing to the social, political and economic development of society.
Given the current economic context and rapidly evolving technology and culture, universities are forced to improve their decision-making processes on a daily basis in order to strengthen their competitiveness and to find the best way to satisfy their users with a more efficient use of resources in products and services.
Consequently, the referential function and assessment actions take on an essential role in this process.The principle of transparency of the new university management model requires the constant self-assessment of universities so that a transparent performance can be ensured and fundamental internal transformations brought about.
However, the evidence-based approach (EBA) is more than just another approach.To date, this movement has been crucial to the new management and decision-making in public education policies.The application of this approach is assumed in order to extract reliable conclusions based on contrasted evidence.In turn, those in charge of decision-making at different levels (political, administrative and technical) are expected to adopt the necessary corrective measures, promoting more rigorous and rational actions for change at different levels of the system.
The use of evidence in informing policies is not a new idea.However, what is an innovation is the increasing importance attached to this concept over the last decade.The increased presence of Evidence-based policy (EBP) was first noted in the United Kingdom in 1997 under New Labour.The main aim of EBPs was to modernise the state while limiting the weight of ideology in decision-making processes, implementing evidence-based policies instead of the policies in place in response to short-term pressures or policies adopted following exclusively ideological criteria.In short, EBPs favoured the use of evidence over that of ideology or opinion in decision-making at a political level.
Two decades ago, Kogan (1995), speaking on behalf of the CERI, highlighted the need for governments and education managers to use the results of education research to encourage rational political decisions in education, as well as processes for change and improvement in educational systems.A decade later, the emerging discourse of evidence was gradually incorporated into the remaining levels of the system: administration and technology.It was at this point that educational language began to use new constructs such as Evidence-based management (EBM) (Pfeffer and Sutton, 2006), or Evidence-based education (EBE) (Slavin, 2002).
In short, the EBA was devised as a systematic and scientific methodological approach on which political, administrative, and executive decision-making could be based to promote the pedagogic regulation of the system itself (D 'Hainaut, 1988), meeting objectives.

Evidence-Based University Teaching: Lights and Shadows
As societies advance, expectations regarding the quality of achievements increase.This is equally applicable to different social systems: the health system is expected to provide quality of life, while the educational system is to provide a high standard of education.
The growth observed in higher education in OECD countries and the need for modernisation of university institutions within the Society of Knowledge has brought about change in the public agenda and interest in the quality of the university system has now replaced interest in coverage.
Quality assessments become particularly important, while also constituting a complex challenge given their multidimensional nature.There is mounting pressure for assessments from different social agents, who are calling for universities to be accountable for the efficacy, efficiency, and quality of the education they provide (ENQA, 2015).
The quality of teaching, as defined in the literature, includes differentiated levels as well as multiple variables, many of which are difficult to quantify.The numerous existing standards and rankings attest to the diversity of approaches, and although these measures are not perfect, they provide relevant information relating to the quality of the institutions.In Europe there is growing interest in outlining a common framework for indicators and standards in the assessment of the quality of university education, respecting autonomy and the responsibility of educational institutions.
Examples of note include the Standards and guidelines for quality assurance in the European Higher Education Area (ESG), passed in the Ministerial Conference of Bergen ( 2005), and the new ESG passed in the Ministerial Conference of Yerevan ( 2015), following a proposal prepared by the European Association for Quality Assurance in Higher Education (ENQA).ESGs were devised as tools for establishing the regulation criteria for internal quality assurance in universities, quality assessment agencies in Europe, and for the nature of external assessment processes.They are also one of the main mechanisms for ensuring trust between higher education institutions and European quality agencies.
The principle of transparency on which European public policies are based also explains the need to disseminate the quality assessments of university teaching in order to inform and cater for the interests of students, employers, and other social and institutional agents involved in higher education.
The main purpose of these assessment studies is to contribute to decision-making regarding the main improvement actions to be undertaken at university level.One example of this is the Report to the European Commission on Improving the Quality of Teaching and Learning drafted by the HLG (2013).This report provides evidence-based recommendations to help the different agents involved in the university system (political, institutional, and individual levels) in promoting transformation and improvement actions within the pedagogic programmes, actions, and devices used to favour the quality of teaching and learning in higher education.
Modern evidence-based public policies and management policies in industrialised countries assume that in the absence of clear and objective evidence to support complex decisions there is a risk that these will be based on the ideology and intuition both of public agents in charge and of other agents with decision-making responsibilities.Due to the increasing socioeconomic complexity and interdependence of public affairs, intuition and ideology alone do not guarantee adequate decision-making.In fact, they also increase the probabilities of decisions being made with insufficient evidence and leading to results that are partially or completely opposed to those sought.In short, the search for evidence to improve the quality of what institutions offer by institutionalising the assessment goes on.
When examining the quality of university teaching, the pros (lights) of the use of evidence are far from simple and univocal, as demonstrated by the many opposing arguments employed (shadows).The same can be said for the arguments advocating the use of evidence extracted from scientific research and compared studies as a useful, valid, and reliable approach in political, institutional, and individual decision-making affecting the quality of university education (UNESCO, OECD/CERI, ENQA). Figure 1 illustrates these arguments. ies.ccsenet.

Lights
The argum quality ma

Shadows
Contrasting arguments can be found to this reasoning.These objections are found mostly among professionals and academics and highlight the epistemological and methodological reductionism of the EBA (Biesta, 2007(Biesta, , 2015)), the limited usefulness of educational research at the service of the improvement of teaching and learning processes within universities (Lingard, 2013;Grimaldi, 2016), and the pre-eminence of managerial use and the assessment and research functions for accountability (Kenny, 2016).A summary of the responses mentioned are grouped below: The hegemony of experimental evidence as a prototype of reliable information.Evidence-based Medicine (EBM) is considered the model of reference for this approach, and at the heart of it lies the rigorous, explicit, and reasoned use of the best evidence available for clinical decision-making.In EBM, the best evidence is that which is supported by reliable, measurable, and observable data obtained using standardised tools, as well as a uniform procedure that can be replicated with controlled or random studies or studies derived from systematic syntheses or reviews of large samples (meta-analysis).This is to say, experimental evidence for calculating the magnitude of the effect of a treatment.Similar examples for research relating to effective teaching methodologies and strategies in university teaching are the meta-analyses of Bernard, Borokhovski, Schmid, Tamim and Abrami (2014), Abrami et al. (2015), Hattie (2015) and Mothibi (2015).This more orthodox interpretation of evidence is contested by pluralist interpretation holding that the evidence achieved through a single research or experimental approach implies the superiority of quantitative studies over qualitative ones.However, most methodologists currently consider neither of these approaches to be intrinsically better than the other, and believe instead that both are part of a continuous process for approaching the study of a phenomenon differently (Hernández-Sampieri et al., 2014).In fact, even the clinical diagnosis of a doctor is based as much on laboratory analysis (quantitative) as on interviews (qualitative), as well as on the review of files containing both quantitative data and qualitative references.The OECD has questioned the orthodox version of evidence and recommends a pluralist interpretation (Burns & Schuller, 2007).
The limited connection of research with teaching practice and the learning of students.The 'top-down' or 'bottom-up' strategies implemented to promote studies and research on the quality of university teaching are useful at political and administrative levels.However, it can be argued that neither the nature nor the results of these studies can be used by the teachers and experts implementing the quality policies promoted from the universities (Murillo-Torrecilla, 2011).It should be noted that this objection is more frequent in countries, which historically have had an enlightened vision of research and the university, as is characteristic of most European countries except the United Kingdom.In contrast, the United States, Canada, and Japan favour a technical or engineering vision.This also explains why teaching does not enjoy the same institutional recognition as research.
In fact, valuing teaching in the same way as research is precisely one of the challenges faced by European universities, with the aforementioned exception of the United Kingdom and northern Europe.
The instrumental use of evidence for the purposes of accountability.Another critical view of the EBA applied to the quality assessment in university teaching defends the importance, in terms of policies and strategies, of setting standards for the assessment of the quality of university teaching.However, in practice, the regulatory use of these standards by universities and national assessment agencies for individual countries has relegated their educational use to the background.In practice, greater transparency and accountability and the instrumental use of evidence as a public decision-making strategy clash with the educational use involved in these standards and evidence.There is a risk of prioritising efficiency and efficacy criteria characteristic of summative assessments and results, over relevant or innovative criteria characteristic of the processual and educational assessments whose effects are unpredictable or not fully known.A recent study carried out in Spain on the opinions held by university teaching staff regarding the quality of university teaching shows mistrust towards the culture and standards of quality used by universities.In fact, the Spanish teachers who took part in this study consider that assessing teaching staff according to the standards for teaching excellence will have almost no effect on the true improvement of the quality of teaching (Barandiaran-Galdós et al., 2012).

Research-Based University Teaching
Closely linked to the EBA in university policy and management, the approach of research-based university teaching begins to gain ground in university discourse.The practice of this approach means that professional experience is combined with the best external evidence available obtained through systematic research.This contrasts with one of the strategies connected with the total quality approach that is part of the philosophy of continued improvement through educational innovation and best practices (Figure 3).

Inspired b conscious, in order to also an org human cap
Among the worth noti efficacy an

Interes
Teachers a educationa skills and the incide modes (bl notable cu profile req In addition education They also members o

Interes
Based on quantitativ costs alloc from syste the new pa with an ac an active approach university outcomes,

Ease in
The g the members of the organisation.Therefore, a collaborative culture must be incorporated into the institution to ensure the efficiency of the KM (Nya-Ling, 2016).Some of the strategies used in the creation and management of knowledge within the organisation are benchmarking, collaborative work, virtual communities of circles of exchange, and knowledge.Under the EBA, encouraging reasoned and effective innovation in university education means choosing a diffuse model for producing knowledge in which teachers play an active role in the creation of knowledge on practice while critically interpreting the best scientific evidence published.

Integrating Scientific and Evaluative Evidence into Teaching
As already stated, the practice of the research-based teaching approach involves the integration of individual teaching mastery and the best available pedagogic evidence, based on the systematic research of individual practice.It is a process of continuous improvement through self-guided, collaborative, and permanent learning in which student learning creates a need to locate information pedagogically relevant to improving the quality of the teaching and learning process to be implemented.In addition, it was observed that the relationship between research and teaching has immediate benefits as regards the depth of learning of students and the possibility of generating academic communities for practice (López-Gómez, 2015).
In short, this is the promotion within university education of the idea of 'teachers researching' their own practice based on the collaborative strategies specified by the approach of Total Quality through didactic innovation and best practice.The Standing Committee for the Social Sciences (SCSS, 2010) and the Scholarship of Teaching and Learning (SoTL) call for the professional training of university teachers through reflection, systematic research of individual practice, and the dissemination of the results obtained within the academic community (Boyer, 1990;Freudenberg, 2012;Lison, 2013).

The Systematic Analysis of Experience: Best Teaching Practices
Given the limitations of the Research-Development-Diffusion-Application in education linear model (Saunders, 2007;Levin, 2011) and the possibilities offered by the research-based teaching approach in improvement the quality of teaching, the Total Quality approach recommends the identification and systematisation of the best practice generated during the organisation process, which can also be used as a model for other organisations.
In general, the concept of best practice refers to any experience guided by suitable principles, objectives, and procedures adapting to specific regulatory perspective or to an agreed parameter, as well as any experience that has provided positive results, proving efficacy and usefulness in a given context.
As regards education, in the framework of the programme Management of Social Transformations (MOST), and according to the glossary compiled by Vlãsceanu, Grünberg, and Pârlea (2007) and Unesco (2003) defines best practice in the following terms: A method or an innovative process involving a range of safe and reasonable practices resulting in the improved performance of a higher education institution or programme, usually recognized as best by other peer organizations (Unesco, 2003).
Best practice is not necessarily an absolute, ultimate example or pattern, nor does its application guarantee the improved performance of a higher education institution or programme.However, it does identify the best approach to a specific situation, as institutions and programmes vary greatly in constituencies and scope (Unesco-Cepes, 2007).
• Representative of a specific situation.

•
Effective.These show a positive and tangible impact on the improvement in real and non-ideal (or laboratory) conditions.
• Sustainable.These can last over time and produce lasting effects.

•
Replicable.These can be used as a model for developing policies, initiatives and actions in other places.
Best practice, systematised, makes it possible to learn from the experience and learning of others and apply it more broadly and/or in other contexts (scaling-up).In addition, it can encourage new ideas or suggest adaptations and provide guidance regarding the most effective way to give visibility to the different effects of the experiences.
In terms of Total Quality, best practice or successful practice are solutions that have been proven to be effective in executing projects, activities, tasks, problem-solving, reaching goals, making the most of opportunities, or creating opportunities that can be replicated.They constitute an excellent strategy for: (a) transforming tacit knowledge into explicit; (b) systematising knowledge derived from the contrasted experience; (c) mobilising knowledge and facilitating innovation.Shulman (1986) holds that the study and diffusion of best teaching practice are major commitments in educational research.We agree with Zabalza-Beraza (2012) who states that best teaching practices should also be researched in university teaching if there is a desire to reinforce their potential as resources for improvement given that: they offer valuable baggage of knowledge and experiences, thorough yet manageable, capable of prompting debate within the scientific and professional community at that educational level regarding the strategies best suited to improving the quality of proposals made to respond to the demand of the different cultural and social contexts (Zabalza-Beraza, 2012, p. 37).

Transferring evidence to practice
One of the difficulties of the implementation of the EBA is transferring the evidence to teaching practice itself.
According to Kirkpatrick (1998), this transfer is a change in behaviour, which brings about permanent improvement in professional practice.This appears to be the key and most difficult point of the practical use of EBA in university teaching.The success of the implementation and development of this approach lies in the encouragement of changes in behaviour in teaching in relation to the available theoretical evidence.In our view, there are two causes for difficulties: • Firstly, because the transfer process does not occur automatically or lineally.This is partly because from the very start it is necessary for the teachers themselves to question and become aware of what they wish to improve or transform in the classroom and how to do it (Georges & Poumay, 2013).In order for the transfer to be effective it must operate on an endogenous rather than an exogenous level.In this respect it is more appropriate to speak of translation or mobilisation of knowledge, rather than transfer in the technological sense of the term.The process involves interpretation, critical assessment, conscious appropriation of the scientific and/or evaluation evidence available, and activation depending on the uniqueness of the education situation that is being worked on (Schwimmer, 2014).At the same time it involves teachers being committed to educational matters over and above the strict limitations of an academic discipline (Mägi and Beerkens, 2015).
• Secondly, because the replicability of a teaching-learning device or a best practice whose efficiency has not been proven does not guarantee the same effects when replicated in conditions different from those in which they were tested.The problem of efficiency as regards the evidence of what 'works' in teaching is not only the difficulty of generalising the results for different locations and contexts to those of the sample selected (external validity), but is above all the difficulty of individualising or contextualising them.In this respect, any attempt to prove the achievements of the process or strategy implemented must also recognise the context, type, objective and recipients of the devices, strategies, or techniques implemented (Feixas et al., 2013).
• Finally, because tacit knowledge, difficult to formalise and transfer through personal interaction and experience has an effect on teaching practices (Oleson & Hora, 2013).It is hardly surprising that there are contradictions between the scientific theories defended and their application in the classroom as demonstrated by Fernández-March et al. (2012) in a recent study on best practice in university teaching in Spain.
In order to ensure the successful development of the EBA the culture of practice-based research must be incorporated into the daily tasks of university teachers with the support of university institutions.The aim of the EBA is to promote the effectiveness of university teaching by integrating research on the practice itself with the best scientific evidence available.Achieving this goal is not exclusively in the hands of teachers.
The implementation of the approach can adopt several strategies and be developed at both institutional and individual level.Important strategies include the dissemination of innovation; encouraging an organisational culture which legitimates teaching (not only research) and research based on teaching; the efficient management of knowledge based on new information and communication technologies; promoting individual and organisational learning (Nutley et al., 2003).In parallel, other authors suggest using local data to show the positive impact of specific educational practices and experiences on the quality of student learning; using practitioner narratives; attaining institutional tipping points and conducting use-inspired research (Cook et al., 2012).
In any case, the processes for transferring knowledge also require an efficient management of teaching research, creating conditions and incentives to make it easily accessible, promoting continued research by external agencies, institutions, and teaching staff, as well as improving basic knowledge of university pedagogy, and legitimating and incentivising basic research geared towards practice.Finally, it is just as important to train teachers to use evidence systematically.

Conclusions
The modernisation of higher education brought about the emergence of a new culture of organisation and assessment based on evidence (Shavelson, 2010).The emphasis on management aimed at results, the need for accountability, the interest in using 'solid' evidence in institutional and pedagogic decision-making, and highlighting the educational function of assessment, help to explain why it is necessary to answer the question what works? in university education.
The architecture of assessment, traditionally limited to politics, has gradually extended into university institutions.With the quality of teaching as a goal, eyes set on the learning goals of students, and the European recommendation of 'encouraging a policy and practice based on real data in education and training' (Council of the European Union, 2009) the culture of evidence began its history with lights and shadows.
Undeniably, the aims which attach significance to the use of evidence in legitimating reform and innovations in university teaching are important.However, any initiatives adopted in the future should be based on the principles of transparency, methodological pluralism, and institutional cooperation and support.Transparency in the clarification and promotion of the standards and criteria should be used to determine the 'solidity' of evidence.Methodological pluralism should be employed when selecting evidence from the diverse existing scientific paradigms and cooperation with university teachers, facilitating suitable educational pedagogic and scientific training when wishing to assign an active and critical role as user and generator of evidence on what 'works', and also what 'does not work' from their own field of influence.Finally, there is a need for institutional support in promoting professional training of teaching in Higher Education from the approach of research-based teaching.

F
anagement in u c regulation.T or pedagogic, re executed, a , analysis, org comparison of ence its proce on for the suita ctions are: (1) blem based on the decision b nt of results to in order to rev ut, 1988, p. 45) ional learning titutions, and t isation of the d nt also plays a 2013).The in o understand t results.

Figure
Figure