Exploring the Effects of Mobile-Based Audience Response System on EFL Students ’ Learning and Engagement in a Fully Synchronous Online Course

Innovative technologies, such as the Audience Response System (ARS) provide an opportunity to steer students into active engagement and meaningful discussions. Many previous studies on the use of ARS, mainly in large traditional classes, have accentuated the positive impact in terms of increased students’ learning and engagement through the incorporation of ARS into classroom practices. However, in synchronous online courses, wherein the lack of visual contact tends to stifle active engagement, the impact of using ARS is certainly worthy of investigation. Thus, in this mixed method study, online English as a Foreign Language (EFL) students’ perceptions concerning the use of mobile-based ARS (M-ARS) and its impact on enhancing their engagement, interactivity, and learning attainment were examined via a questionnaire whereas challenges pertaining to the use of M-ARS were solicited via semi-structured interviews. The results revealed that the implementation of M-ARS in online teaching correlate significantly with EFL students’ engagement and learning experience whereas qualitative analysis revealed some important points with regard to integrating M-ARS into online classrooms. Directions and suggestions for future research are offered.


Introduction
In recent years, mobile phones have evolved from merely being devices for voice communication to multifunctional sophisticated digital devices with ever increasing processing and storage capabilities.The benefits of utilizing mobile phones to support language teaching and learning, both inside and outside classrooms, appears to be their affordability, flexibility, portability, user-friendliness, and multifarious built-in features.Not surprisingly, the benefits and appeals of mobile phones have brought up a plethora of research studies that investigated how these smart devices can be effectively incorporated into language teaching and learning both in EFL and ESL contexts.Over the last two decades, mobile-assisted language learning (MALL) has emerged as a field in its own right in the field of English language teaching (ELT) (Stockwell & Hubbard, 2013).MALL, as defined by Kukulska-Hulme (2013), is the use of "mobile technologies in language learning, especially in situations where device portability offers specific advantages" (p.3701).Despite being a 'fresh' research field, a review of MALL research from 1994 to 2012 by Burston (2013) revealed in excess of 575 MALL research studies.As smart phones and other hand-held devices are unceasingly developed, it is believed that their integration in mainstream education as well as in EFL/ESL classrooms will only continue to rise.
As providing feedback on students' learning has always been a central pillar of students' language learning and consolidation, educational mobile apps have increasingly become essential to support language teaching and learning in ways that are deemed unfeasible.In this regard, M-ARS is a recent innovative mobile technology, which allows English language instructors to pose questions to EFL/ESL students during class time, to which they are able to instantly respond via their phones.Students' responses can be analyzed rapidly and displayed in informative graphs and charts for all learning/teaching participants.Traditional ARSs, commonly referred to as clickers, have been utilized in higher education for more than a decade to date and many instructors at North American and European colleges and universities have adopted ARSs in their classroom teaching practices (Han & Finkelstein, 2013).As smart phones have become increasingly ubiquitous and internet accessibility more affordable, M-ARSs have gradually begun to replace the traditional clickers.M-ARSs are an effective and viable alternative to "clickers", which have frequently been criticized for being expensive, difficult to use, complex to manage and initially setup to operate, and require regular maintenance (Caldwell, 2007;King & Robinson, 2009).
Recently, there seems to be an increased interest among researchers in the examination of the efficacy of using M-ARS for supporting and monitoring students' learning, and promoting collaboration and engagement among students (Awedh, Mueen, Zafar, & Manzoor, 2014;Dervan, 2013, Voelkel & Bennett, 2014;Dunn, Richardson, Oprescu, & McDonald, 2013).These studies' findings conclude favorably concerning the use of M-ARS in traditional education.In online distance education, however, the use of M-ARSs is relatively a new practice.Thus, the current study aims to investigate the EFL students' perceptions concerning the use of Socrative, a free mobile-based ARS, on their learning interaction in a synchronously delivered English online course in a Saudi Arabian university.

The Efficacy of Traditional ARSs
The bulk of the research on ARSs has examined the efficacy of these devices in traditional classrooms.The majority of this research has demonstrated positive impacts of ARSs on enhancing learning outcomes (Voelkel & Bennett, 2014), bolstering students engagement (Kotevski, 2014;Dunn, Richardson, Oprescu, & McDonald, 2013), improving students' attendance (e.g., James & Willoughby, 2011), stimulating peer and in-class discussion (Pelton & Pelton, 2006), and encouraging students to believe that these devices can aid them in identifying misunderstandings of learning concepts (Bode et al., 2009).Research has also revealed that the use of ARS in classrooms has been inextricably linked to students' emotional, motivational, and cognitive classroom experiences (Simpson & Oliver, 2007).
Despite growing interest in ARS, little research has been devoted to untangling the mechanisms through which these systems influence students learning outcomes.Blasco-Arcas, Buil, Hernández-Ortega, & Sese (2013) emphasized the need for having a thorough understanding of the underlying forces that make these systems beneficial to the enhancement of student learning.Drawing upon theories of student behaviour and learning and on previous empirical research, the researchers have developed a conceptual framework in which interactivity, active collaborative learning and engagement are hypothesized as influencing students' learning outcomes.To test their proposed conceptual framework, the researchers have examined the perceptions of 198 undergraduate students on the use of ARSs via a questionnaire.The results obtained have provided a strong empirical support for the proposed model as they have demonstrated that students' perceptions of the interactivity with peers and teachers as a direct result of ARS use promoted active collaborative learning and engagement, which ultimately led to improved learning performance.
As appealing as the benefits of using ARS seem on face-value, they are often criticised for posing some difficulties and practical challenges that may severely hinder their successful implementation in classrooms.One of the oft-cited challenges of traditional ARS is their high cost.These hardware devices are expensive and often unaffordable for students or educational institutions.Another issue is that both teachers and students need to be trained in the use of such high-tech machines (Caldwell, 2007).The management of these devises and their setups are also complex (King & Robinson, 2009).

The Efficacy of M-ARSs
The use of is a relatively new practice in higher education.The recent technological development that mobile phones have undergone and their ever-presence in the hands of today's learners have made the possibility of utilizing them in classrooms a viable alternative to traditional ARSs.In a recent case study, Voelkel & Bennett (2014) have assessed the viability of using students' mobile phones in combination with text messaging as a polling system.Using their smart phones, students in four large undergraduate biological science classes sent their answers to formative assessment questions and their answers were instantly displayed on a PowerPoint presentation.The study has also evaluated how students perceived the use of mobile-based polling system via an online questionnaire.The results have shown that students viewed this new technology as beneficial and the system had helped them to identify their strengths and weaknesses.In addition, an improvement in students' performance as a result of using M-ARS is clearly discernible.Awedh et al. (2014) have examined the impact of using Socrative ARS on supporting collaborative learning among 38 undergraduate students in a community college in Saudi Arabia.The students used Socrative via their smartphones to answer questions pertaining to their classroom tasks.The findings have revealed that using Socrative promoted students' collaboration, engagement and motivation and supported their learning.
Similalary, Dunn et al. (2013) have conducted a study to explore the benefits of a low-cost mobile-based ARS, VotApedia, with 731 undergraduate students from health, science and engineering disciplines.Data were collected via an online student survey and students' feedback on the course and teaching was sought in two focus groups.The results have indicated that using M-ARS retained the majority of the benefits of traditional ARSs, overcame some of the challenges of traditional ARSs, and yet introduced very few new challenges.One such challenge was that not every student had a mobile phone, whilst another was that other students had their phones with them, but could not use them for various reasons such lack of sufficient credit.
Educational research has accentuated the salutary impact of teachers interlacing their teaching practices with frequent interactive, timely, and meaningful formative feedback on their students' learning and performance (Gibbs, 2010).However, in online classes, it is impractical for teachers to provide such valuable feedback to individual students.In synchronously delivered online courses, engaging students and providing timely feedback on their learning progress is effectively prevented by the lack of teacher-student face-to-face visual contact.Subsequently, online teachers often fail to monitor or gauge their students' understanding of concepts taught.Our review of the literature revealed that no prior study has examined the impact of M-ARS on students' learning and engagement in online language teaching and learning, particularly in Saudi EFL context.Thus, with an aim of addressing this gap in the existing literature, the present mixed method study examines EFL online learners perceive the use of their mobile phones as ARS while attending synchronous tutoring sessions in a Saudi Arabian university.Specifically, this mixed-methods study intends to answer the following two primary research questions: 1) What are the EFL students' perceptions about the use of M-ARS as an integrated part of their online course and how do they perceive the impact of M-ARS on their engagement and learning in an online course?
2) What challenges do EFL students encounter while using M-ARS in their online classes?

Method
This small-scale study has adopted a mixed-method approach, the process of mixing quantitative and qualitative methods of research.According to Collins, Onwuegbuzie, & Sutton (2006), the mixed method approach involves the following iterative steps: determining the mixed-method goal of the study, transforming and analyzing the quantitative and qualitative data, and legitimizing the data collected.Onwuegbuzie et al. (2009) gave a three-dimensional presentation of the mix-method involving; a) case-oriented which applies to one person and a tendency towards generalizations, b) variable-oriented which refers to variable entities, and c) process-oriented which involves the experiences of one or more cases.
To explore the EFL learners' perceptions of Socrative ARS, mix-method approach fitted the purpose of this study mainly due to the following five reasons identified by Collins et al. (2006): a) to improve the accuracy of the data on teachers' identity; b) to produce more complete picture by combining information from the quantitative and qualitative data; c) to avoid the biases of intrinsic single-method approach (quantitative); d) to develop in-depth analysis on the initial data, and 9) to use questionnaire to screen the potential participants for inclusion in the interview process (pp.79-79).

Context of the Study
This study was conducted in an English language institute of a Saudi Arabian university.Socrative, a free mobile-based ARS, was integrated into the teaching of an online introductory English 101 course in spring 2014 and fall 2015.The courses were taught by one instructor for 12 weeks, a synchronous hour-and-twenty minutes teaching session per week.The EFL instructor introduced the Socrative mobile app to all the EFL students in the first session and asked all of them in a voluntary manner to download the app from the Google Play © store or from the Apple App © store for those with Android © or iOS © mobile devices respectively.Before the course time, the instructor created a short formative assessment quiz covering major teaching points in each teaching session.The quizzes were in form of multiple-choice or true and false types of questions.EFL students accessed the quizzes via Socrative mobile app while attending their online sessions.The instructor received a real-time response on students' engagement and performance and provided constructive feedback on EFL students' answers (see Figure 1).Reports of EFL students' performances were collected and downloaded as Microsoft Excel © spreadsheet.ijel.ccsenet.

Partici
A survey q collected t 10 times w using the a co-educati years, with In the second cycle of coding, identical codes were grouped together in order to identify indistinguishable patterns in the data.This process was followed by the categorization of the data.Finally, six main categories occurred which resulted in the development of three overarching themes, discussed the following section.

Quantitative Results
The first step taken in the data analysis was to test the internal consistency or reliability of the fifteen Likert-scale items in the questionnaire.This was carried out using the Cronbach's alpha test on the reflective assigned to each response.The calculated value of the Cronbach's alpha of 0.785 indicated that the questionnaire is consistent and reliable to be utilised for collecting data from the participants.

The Relationship between M-ARS Implementation and Students' Perceived Effects on Engagement and Learning
A Pearson's correlation analysis was conducted on the EFL students' use of M-ARS and their perceptions of the effects of M-ARS implementation, and on their perceived engagement as the result of using M-ARS, and on their learning experience.Results, as shown in Table 2, indicate a significant correlation between the implementation of M-ARS and students' engagement (r= 0.743, p< 0.01).There was also a significant correlation between the implementation of M-ARS and the EFL students' learning (r= 0.757, p< 0.01).

Qualitative Results
This section of the paper presents the qualitative findings of the study generated through semi-structured interviews.The inductive analysis of the data has led to the emergence of three major themes; Socrative app as a learning tool, app associated issues and EFL learners' attitude toward phone application.These themes are further explained in the following paragraphs.

Socrative App as a Learning Tool
The learners' interviews revealed that the use of the Socrative app as a language learning tool was a beneficial experience for them.It is evident from their responses that they utilized the opportunity and gained exposure to the app through online classes.They liked the idea of using the app as it enabled them to interact with technology and made them learn in an unconventional way.The below extract from the interview of student 1 provides an insight into how students found the experience of using the Socrative app.

The app is great and its concept is really good. It gives me an idea of the content of the lecture and a good summery of the key points of the lecture through a short quiz (Student 1).
The learners also found the Socrative app helpful in organizing their notes, points of lecture and key ideas for the quizzes.Moreover, the students considered it an effective source for obtaining prompt feedback from instructors on their work while working on their tasks in online classes.
Knowing the answers after doing the quiz is really helpful.The teacher knows his students' performance and he can identify who is excellent and who is good and who is struggling.If the teacher wants to select his best students he can because the students who did the quiz fast are good students, we had a short time.

In the class I know the students who are good (Student 3).
This assertion also indicates that teachers can use the app in order to check learners' comprehension.In cases where learners find it difficult to answer particular questions, this means more clarification is called for, which enables teachers to use the app as a diagnostic tool.
The data indicated that though the app might be used anywhere as it could be downloaded on smartphones, the learners would prefer to use it as a formative assessment while attending their online classes.In terms of classroom learning, the app not only made the lessons interesting and interactive, it also proved very constructive in the process of taking quizzes.The students were of the view that prompt feedback on their quizzes helped in their learning.
The app received appreciation from the learners; however, the majority of them did not consider their smartphones a useful learning tool in this regard.They believed that the app might be utilized in a positive way, however there was no guarantee that all the students would make good use of their phones.For example, students could abuse the online facility and take it for granted while answering the quizzes.
Doing the quiz during the class would be difficult for some weaker students with low English proficiency especially in writing as we have one hour and we can open the book and then answer the questions to assure that we get 30/30 in the quizzes (Student 2).
As the data revealed that the app had certain features that could help learners in improving their language skills, it was certainly dependent on learners' own attitudes to turn the app into a learning device.

Learners' Attitude toward Socrative App
The data indicated that learners had various preferences regarding the use of the Socrative app.For example, the learners acknowledged the app's availability on a smartphone as they could access and use it anywhere.However, they preferred to use it on their computer while browsing webpages as indicated by student 5, "I connect my laptop to monitor but whenever I travel I use my phone".According to them, the computer screen helped them focus more on the contents on the app and they were able to browse other relevant pages simultaneously.

The phone is small and the words should be large to see them and sometime the apps work slowly or gets frozen on the phone and honestly, I rarely did my quizzes via my phone (Student 5).
The learners also showed their will to use the app frequently while attending their online lessons.As they considered it easy to use, convenient and user-friendly that connected them to their classroom learning.The extract from student 3 explains this point.
I used my phone at the beginning and then I used the web later.I feel the website is more convenient, everything on the same page, and on the same computer, and it's better in use (Student 3).
The findings showed that the learners used the app frequently with the intent to learn and improve.During the online lesson, they could use other applications installed on their phones, i.e., WhatsApp etc., however, they focused on their lessons and maintained their interest in their classroom.The learners' seriousness and their intrinsic motivation were obvious factors that enabled them to effectively use the app and enhance their language skills.

I never use my phone while attending my class because to me the lesson is very important and I need to pay very good attention to the instruction in the class (Student 1).
The learners' interest in the app was not limited to the existing English language courses; rather they wished to have such apps for other courses as well, which would keep them immersed in their studies and help them overcome their learning obstacles.
I think the idea is great and we should use the app in our other courses as well and we should have used the app in our previous courses, from the beginning of the year (Student 4).
Despite its various positive learning aspects that the learners pointed out, they also associated certain difficulties with the app, which they would like to see remedied.

Socrative Associated Issues and Challenges
Although most participants found the app a convenient and user-friendly tool, they thought it could be difficult to operate at the start especially when no prior training had been provided.It was clear from the learners' responses that none of them found it easy to use it at the beginning as no user-guide was provided for them.They learnt it by exploring it through trial and error, a kind of self-learning approach.Thus, a little training would help students to effectively use the app rather than getting side-tracked with technicalities and technical issues that might divert their attention from the lesson.
There are various issues that might have posed challenges to the learners in their bid to use the app in their online lessons.The most common one was the English version of the app as students found it difficult to interact with the app in English and their preferred way would be for it to be available and accessible in Arabic.Google translation did not always help them.In addition, they thought that different interfaces of the app also made it slightly difficult for them to use it in a professional manner.Although, the app interface is simple and easy to follow, student 5 came up with an unexpected comment as he said: The app might have many interfaces as you open it you find many icons; however, it's simple to use.In addition, it's in English.If you want to use it with weaker students you should make it in Arabic because Arabic is our language (Student 5).
To neutralize these issues, the data suggested that the learners should be given an opportunity to develop familiarity with the app.The learners thought that prior training would be a great opportunity to interact with the app and would to a certain extent negate fear of failure in quiz preparation.Furthermore, they expressed the concern that the lack of familiarity might cause increased levels of frustration during quizzes.The data also revealed that students often lacked confidence at the start mainly due to lack of exposure to such apps and poor language proficiency.
If there was a guide on its use, it would be very helpful.I seriously did not know how to figure out all the information teacher gave us like there was a code to enter the online room etc.I really got confused, really the website is confusing and needs more simplification (Student 2).
The learners showed their interest in the app, however, they would love to see more simplified, possibly in Arabic language interface accompanied by a user guide.This can be attributed to the students' lower English language proficiency.

Discussion and Conclusion
The use of new and affordable technologies in the teaching and learning of second language is highly recommended and many ESL/EFL instructors are continually in the search of new ways to engage their students in the learning process.One of the challenging aspects of EFL/ESL teaching online is keeping students engaged and involved in the learning process to enhance the learning outcomes.M-ARSs is a recent innovative technology that can have positive impacts on online learners' engagement and interactivity.
The current mixed-methods study has revealed some important issues pertaining to the use of M-ARS as a formative assessment tool in a fully synchronous online English course.One major finding is that the EFL students found using M-ARS a helpful tool in their learning as it leads to increased levels of interactivity and engagement.This finding corroborates previous research on ARS, both traditional ARS and web-based or mobile-based ARS (e.g., Awedh, Mueen, Zafar, & Manzoor, 2014;Dunn et al., 2013;Dervan, 2013;Smith & Cogdell, 2010;Voelkel & Bennett, 2014), in suggesting that M-ARS can play a role in encouraging EFL students to be more engaged in the process of English language learning.The EFL students appreciated the use of the app to check on their understanding on what they had just learned.They also appreciated receiving constructive feedback and further explanations from their instructor on questions or learning points that they had found difficult to understand.These findings are of immense pedagogical importance for synchronous online teaching, as it is difficult to check on EFL students' understanding and to keep students engaged in this mode of online delivery.The use of M-ARS is also beneficial for online instructors to use the "just in time teaching" approach relying on live feedback and performance reports that M-ARS provides on students' learning to address their misunderstanding (Marrs & Novak, 2004).
Although students received using their phones as ARS and showed a strong interest, their registration into the system dropped significantly from 84% at the beginning of the course to 12% at the end of the course.This may be due to the fact no marks or rewards were given for participation.Other students opted not to participate due to their fear that their instructor might have a negative impression of their level of understanding while performing poorly on the quizzes.Voelkel & Bennett (2014) reported simialr findings in their study on the use of M-ARS with face-to-face students.Indeed, the live reposnse of the M-ARS may help EFL instructors to have a good indication of students' understanding, but it may add pressure on EFL students when they need to answer difficult questions in a short time.
While M-ARS may be useful, some degree of caution is needed as EFL students may easily get distracted when using their mobile phones.One student from this study reported that he stopped using the M-ARS and went on to use other apps, e.g., WhatsApp and Twitter.This situation may easily arise with teaching distance learning students, given the lack of face-to-face visual contact in online classes.

Limitations and Direction for Future Research
This study is a part of a larger research project, which examines the efficacy of M-ARS in formative and summative assessments in online EFL courses.Future research can look into the impact of different factors, e.g., EFL students' engagement and interactivity with peers and teachers, and their metacognitive skills, on EFL students' use of M-ARS in online classes.Moreover, EFL teachers' perceptions can be explored to understand

Table 1
based or visual data".The first cycle of inductive analysis of the qualitative data led to the emergence of 35 open codes.

Table 2 .
The results of pearson's correlations Correlation is significant at the 0.01 level (2 -tailed).