Instructional Efficacy of Portfolio for Assessing Iranian EFL Learners’ Speaking Ability

Regarding the fundamental role of speaking in language skills, this study intended to investigate the effects of speaking portfolio as an alternative form of assessment for assessing Iranian EFL learners’ speaking ability at the intermediate and advanced proficiency levels and also its impact on their attitudes. Accordingly, from the population of 72 students studying at Kowsar Language Institute in Esfahan, a sample of 64 male and female intermediate and advanced students were randomly selected based on their scores on an OPT test and they were assigned to 4 groups: intermediate and advanced experimental groups and intermediate and advanced control groups. In order to collect the data, a pretest and a posttest as well as a questionnaire were employed. To analyze the data, an ANOVA and a series of Chi-square were run in the study and the findings indicated that the experimental groups using speaking portfolios performed better than the control groups in terms of speaking ability. Moreover, the result shed light on the advantages of speaking portfolios such as self-assessment, peer-feedback, and improvement of speaking skill. This study provides instructors, administrators, and test developers with alternative ways to improve and assess speaking skill through speaking portfolios.


Introduction
To meet the needs of second language (L2) learners learning English, well-established assessment procedures are imperative (O'Malley & Pierce, 1996).It is believed that "L2 learners are best served when teachers recognize when to shift from traditional assessment to learner-directed assessment."(Alan & Pierson, 2000, p. 105).Brown (1998) argues that because L2 learners bring different learning styles in the process of L2 learning, standardized tests have shifted to alternative assessment over the past two decades.He notes that this change is useful because it involves L2 learners in the assessment process.1).It applies records of L2 learners' work over time and in different modes to demonstrate the profundity and progress of L2 learners' abilities.
2).It is a collection of L2 learners' work which is purposeful and systematic and reflects learning achievement based on the objectives of instruction.Jongsma (1989) states that the notion of portfolio comes from the arts in which art students keep evidence of their own work to show their achievement.Portfolios are practical ways to accumulate work of learners and to evaluate their progress regarding the objectives of the curriculum.Valencia (1990) states that portfolios are used to show students' achievement in classroom teaching.However, the portfolios in classroom may focus on progress, formal products, and rating learners' performance considering specific objectives.Little (2009) mentions that as part of the portfolio process, students are asked to think about their needs, goals, weaknesses and strategies in language learning.They are often asked to select their best work and to explain why the work is valuable to them.When the students prepare their own portfolios, they self-monitor their performances and utilize their own learning portfolios to do learning assessment and self-correction.Portfolios also allow learners to see gaps in their own learning and take some risks (Ekbatani & Pierson, 2000).Portfolio can be in different kinds including paper portfolio (traditional one), technology-based portfolio, electronic portfolio (e-portfolio or digital portfolio), video portfolio, and audio portfolio.As Chang, Wu, and Ku (2005) note, portfolios have been used for different skills in the field of English as a Foreign Language (EFL).Danny Huang and Alan Hung (2010) set out a twofold study first, to investigate if e-portfolios affect language quantity, lexical richness, and syntactic complexity of EFL students' oral performance in conversation classes and second, to examine the students' attitudes towards the employment of this digital device as the medium of oral work assessment.In the research, control and experimental groups were administered a pretest and a posttest with the use of e-portfolios.The results suggested that the e-portfolio group performed better regarding language quantity, use of various lexical items and vocabulary.Also, the subjects of the research had positive perceptions towards using these types of portfolios, as they reported that the advantages to the e-portfolios outnumber the disadvantages.The researchers claimed that e-portfolio enables students to revise and resubmit their artifacts as many times as they satisfy therefore they monitor their progress over time.Castaneda and Rodrigues-Ganzález' (2011) study examined students' positive attitudes to oral portfolio employing video recording.The finding indicated that 80% of the participants voiced the effectiveness of portfolio in terms of self-reflection.Moreover, participants pointed out that they ameliorated in their speaking performance using portfolio.Similarly, Bolliger and Shepherd (2010) conducted a study on learners' perception of adapting e-portfolio into classes to improve oral skills.According to the results, the researchers suggested that 89% of the students arrived at the positive role of portfolio in self-assessment as well as speaking.Besides, Wang and Chang (2010) analyzed the effects of self-monitoring and portfolios on college students' anxiety levels and oral skills.The results revealed that oral portfolio allowed students to learn not only from themselves, but also from their peers and their teachers, thus the students could self-monitor and change their learning process, consequently (Wang & Chang, 2010).Further, the researchers explained that due to the use of such an alternative assessment, students' communication apprehension would be decreased.
In addition, Brook (1999) carried out a study on speaking portfolio with 86 students from 9 th , 10 th , 11 th and 12 th grades to compare the control and the experimental groups in various levels.The experimental group was assessed through students' speeches which were videotaped and also the feedbacks of those students on their own speeches whereas the control group was assessed traditionally by the teacher after the speech was presented.The results revealed that the experimental group using video-portfolios performed better than the control group.Besides, the subjects of the research had positive perception towards portfolio assessment compared to traditional assessment.
Because speaking is the most difficult skill to be assessed (Chang, Wu, & Ku, 2005), Cataneda and Rodriguez Gonzalez (2011) suggest that probable difficulties in evaluating speaking including subjectivity or affective problems like anxiety and motivation could be solved through portfolios.Another problem in oral assessment suggested by Chang et al. (2005) is the authenticity of the environment.As a result, the present study intends to serve as a study of portfolio implementation on improving students' speaking skill, and it aims at finding out the students' perceptions towards the effects of speaking portfolio.The present study might add a new dimension to the literature by revealing the effects of using video recordings to assess L2 learners' oral performance.Therefore, the findings may be useful to determine the strong points of the video portfolios in different educational settings.In particular, the findings of the study can provide administrators and instructors with up-to-date information to conduct oral assessment.Consequently, discovering different opinions might help program designers and test developers with greater insight into alternative assessment, materials, and curriculum change for oral performance.
According to (Chang et al., 2005;O'Malley & pierce, 1996) students can collect, store and keep these portfolios in a relatively efficient way compared with their paper based counterparts.Such an authentic spoken language assessment makes students more active and also teachers more responsible (Castaneda, & Rodriguez-Gonzalez, 2011).In essence, by considering the apparent lack of appropriate speaking assessment, analyzing advantages of speaking portfolios as an oral assessment could shed light on the way towards preparation of successful oral assessments.
In order to accomplish the objectives of the study, the following questions are addressed: 1).To what extent do video portfolios improve the intermediate EFL learners' speaking skills?
2).To what extent do video portfolios improve the advanced EFL learners' speaking skills?
3).Are there any advantages in using video portfolios as an alternative form of assessment in EFL classes based on attitudes of Iranian intermediate and advanced EFL learners?

Participants
To carry out the investigation, 72 male and female Iranian intermediate and advanced EFL learners, aged 14-18 took part in the study.The rational behind this was that these students had already passed conversation courses.All the learners were chosen from Kowsar Language Institute in Esfahan owning to the availability and accessibility of the data.In order to monitor the participants and to ensure the homogeneity of the participants in terms of proficiency, an Oxford Placement Test (OPT) was used.Based on the results, 64 students were randomly chosen to take part in the study and they were divided into four groups: experimental and control intermediate learners as well as experimental and control advanced learners.Then, 15 of them took part in the pilot-testing of the students' questionnaire in order to check and ensure the usefulness, clarity, relevance, format, and time allotment of the items.Both experimental and control groups were taught under very similar condition and the only emphasized difference between them was alternative assessment using speaking video portfolios received only by experimental groups.Consequently, the experimental groups were asked to answer questionnaires after the implementation of speaking video portfolios.Besides, a group of eight Iranian female EFL teachers participated in this study in order to provide help by teaching the students in both the experimental and control group.

Instrumentation
In order to collect the data, the following instruments were used in the study: 1) Oxford Placement Test, 2) video portfolios, 3) oral proficiency pretest and posttest, 4) a questionnaire, a checklist, and an overall reflection paper prepared by implementing the questions in the study of Özdemir-Çagatay (2012).It should be mentioned that the reliability and validity of the questionnaire was checked (Coronbach Alpha Coefficient was found as .935).

Materials
Two textbooks were used in the study: 1) Tactics for Listening Development, Third Edition, by Jack C. Rechards, for intermediate level students and 2) Tactics for Listening Expanding, Third Edition, by Jack C. Rechards, for advanced level students.

Procedure
After finishing the sampling procedure, the data collection procedures were carried out in quantitative and qualitative phases.First, the selected participants took the pretest in order to evaluate their oral proficiency in 15 minutes and it was comprised of a two-minute personal introduction, five-minute pair-work dialogues, and three-person group discussions with given topics.The speaking class included 20 sessions and was held three times a week.The textbook was Tactics for Listening Development and Tactics for Listening Expanding and 12 units were covered.A group of eight Iranian female EFL teachers were introduced to speaking portfolios, their components, aims, and functions and also they were trained to use speaking portfolios to the experimental groups (intermediate and advanced) by showing the requirements for the portfolio without grading the students' first performances.Each session the students were assigned with an individual, pair or a group work tasks taken from the course book inside the L2 classrooms with the teachers' assistance.Then, the teachers recorded the students' speeches every session using video cameras and uploaded them on CDs or DVDs.After 3 units, the teachers displayed the recordings to the class to enable them to give feedback on their own performances with the guidance of a checklist (see Appendix A).Additionally, the students gave feedback to each other with the help of teachers and by leading them with some questions.The checklist contained both the self-assessment and the reflection part for each unit and it also had a part for the teacher evaluation at the bottom (see Appendix A).The students were asked to assess their own performance based on the checklist and write a reflection by responding the guiding questions of overall reflection paper (see Appendix B).Furthermore, the teachers graded students' performance as well as their written reflections.
The control groups (intermediate and advanced) were taught using interaction in pairs, groups, and with the topics determined from the course book the same as the experimental group.The only difference between the experimental group and the control group was the matter of video recording of students' interactions inside the classroom as well as students' writing comments on their own process.
At the end of the term, the posttest was run in 15 minutes to compare the differences in oral proficiency between four groups of participants.To assure the reliability of the scoring, one other scorer scored the pretest as well as the posttest and inter rater reliability of both tests were calculated as .91 and .94.The students were also given the questionnaire (see Appendix C) in order to identify their attitudes on the effectiveness of speaking video portfolios which took 15 to 20 minutes.

Data Analysis
After the essential data were collected, the Statistical Package for Social Sciences (SPSS) was used in order to run required statistics.Both quantitative and interpretative approaches were adapted to the data analysis to be able to make claims about the efficacy of speaking portfolios for speaking assessments of the intermediate and advanced EFL students.Descriptive statistics (minimum, maximum, mean, standard deviation, skewness, and kartosis) and a series of Chi-square tests were first computed for all the parts of the students' questionnaires used in this study except the last four open-ended questions.The last four, open-ended questions on the questionnaire were examined through deductive and inductive qualitative analysis techniques.As for the deductive analysis, the data were analyzed via categorization of responses depending on the categories in the survey.Besides, the deductive analysis, inductive analysis of the qualitative data was also employed.In particular, the data were examined depending on the recurring reasons for students' preference of the speaking assessment.Moreover, analysis of variance (ANOVA) was run to explore the statically significant differences between experimental and control groups in advanced and intermediate levels in terms of their speaking abilities improvement.

Results of the Pretest
In this section, first the descriptive statistics of the pretest including the mean, standard deviation and 95% confidence interval for the dependent variable (speaking) for each separate group (advanced experimental, intermediate experimental, advanced control, intermediate control), as well as when all groups are combined (total), were calculated to consider the central tendencies and variability of the scores.The results indicated some difference among the means.In order to determine if this difference was statistically significant, an ANDVA was run.Table 1 summarizes the results of the ANOVA.The amount of significance level (p= .557)observed in Table 1 revealed statistically an insignificant difference among the groups.Therefore, it can be concluded that the control group and the experimental group at the intermediate and advanced level were homogeneous in terms of speaking.

Results of the Posttest
To evaluate the subjects' oral proficiency after the study, a posttest was carried out.Table 2 shows the descriptive statistics of the posttest in case of speaking after the implementation of speaking video portfolios.As Table 2 demonstrates, there was some variation among the means.To find out if this variation was statistically significant, the ANOVA was done.Table 3 summarizes the results.Considering the result of the ANOVA, shown in Table 3, it was perceived that, the difference among the groups (control and experimental) was statistically significant at the value of (P= .00).Therefore, the researchers came to the conclusion that the control and experimental groups were different with reference to speaking.This conveys that there was a relationship between the groups and treatment (speaking video portfolios).To see for which group this difference was significant, a Scheffe Test was done.Table 4 shows the result of the test.Result of Table 4 indicates that the difference was significant in experimental group since in this group the scores had increased, while in control group this was not the case.

Results of chi-square
Through this stage of the data analysis, all Likert-scale items of the second part of the questionnaires were assorted regarding the categories constructed based on the literature review such as peer-feedback, self reflection, and speaking skill improvement.In the next step, the frequency, percentage, chi-square, difference, and significance level for each item was computed to indicate students' positive attitudes with reference to each category and also to convey the existence or nonexistence of the relationships between the variables investigated.The items were assessed categories ranging from 1 to 6.The scores for the responses were such as: strongly disagree=1, disagree = 2, partially disagree = 3, partially agree = 4, agree = 5, and strongly agree = 6.Table 5 summarizes the analysis of the results.The results of the chi-square display that the overall attitudes of the participants towards speaking portfolio were positive.Also in respect of self-reflection, most of the participants believed that speaking portfolio provided them with the opportunity to evaluate themselves.Additionally, concerning peer-feedback the majority of the participants agreed that the error correction done by their classmates enabled them to recognize their own mistakes.Moreover, when it comes to the participants attitudes towards speaking portfolio in relation to speaking skills, most of the participants mentioned that speaking portfolio contributed to their speaking skill.

Results of the Last Four Open-ended Questions of the Questionnaire
The last four open-ended questions of the questionnaire focused on whether students like the speaking portfolio or not and also if they prefer traditional exams to portfolios as an assessment or conversely.The data were coded based on the responses of students for each of the aforementioned categories in questionnaire.In particular, the data were analyzed according to the recurring reasons for students' preference of the speaking assessment.The results could be seen in the following part.
The majority of the students had positive attitudes towards speaking portfolio and noted that they could easily see their development in speaking via speaking portfolios (self-assessment).Here are some comments: "By speaking portfolio, we can see development in our speaking and evaluate ourselves.I do not think that traditional exam show our progress exactly."(student 14).
"I had an opportunity to evaluate myself on my own."(student 5).
"Speaking portfolio enabled me to easily see my progress."(students 8).
There were 5 students who pointed to the issue of peer-feedback for the reason of liking portfolio by believing that speaking portfolios helped them to recognize their own mistakes (peer-feedback).Some of the ideas were presented here: "Speaking portfolios helped me to learn from my classmate and to understand my own mistakes."(student 4).
"My classmates are able to watch my performance and help me to correct it."(student 12).
Moreover, slightly more than half of the students preferred the speaking portfolio to traditional exams and stated the positive effect of speaking portfolio on their speaking skills.Most of them pointed positively to having the opportunity to speak in public or practice speaking in general and some also remarked the improvement in the organization of their speech as well as pronunciation skills.The following part shows some examples: "It helped me to speak English a lot." (student 11).
"It improved my speaking and helped me to speak in an organized way." (student 18).

Discussion
Referring to the first research questions of the study which asked if video portfolios improved the Iranians intermediate EFL learners' speaking skills, the result of ANOVA as well as the difference between mean scores of the students' in pretest and posttest pointed to the fact that the intermediate students in experimental group performed better in terms of speaking skill than the intermediate students in control group after the treatment of speaking video portfolios.This result is in line with Danny Huang and Alan Hung's (2010) study, in the sense that control and experimental groups were administered a pretest and a posttest with the use of e-portfolios and the results suggested that e-portfolio group shaped their progress regarding speaking over time.Based on the present finding, speaking portfolio had a pivotal role in improving the intermediate students' speaking skills and as a result there was a significant relationship between speaking video portfolios and improving the intermediate students' speaking skill.
Considering the second question of the study, which investigated whether video portfolios improved the Iranian advanced EFL learners' speaking skills, as the result of the ANOVA as well as pretest and posttest indicated, the advanced students in experimental group performed better than the advanced students in control group in terms of speaking skill.This implies that through alternative assessment using speaking video portfolios, students monitored their speaking progress more than they could in the traditional achievement exams.This finding overlaps with the findings of Brooks (1999) as the experimental group using video-portfolios performed better than the control group regarding speaking.Furthermore, examination of the checklists in which the experimental groups have given feedback on their own performances, revealed that via implementation of alternative assessment (speaking portfolio), students analyzed how well they had learned and what they needed to work on next.This shows the explicit and significant difference between the alternative assessment (portfolios) and traditional assessment (paper and pencil exams).As portfolios invite students to evaluate their performances as a part of developmental process rather than an end point.
One of the outstanding results obtained by this research was that the speaking portfolio is mostly culture based.This means that the male students felt more comfortable to be video recorded than the female students.The researchers assumed that this issue directly refers to Iranians' culture as generally males feel less tense to speak out than females.
To address the third question of the study, which attempted to find if there were any advantages in using speaking video portfolios in EFL classes based on the attitudes of intermediate and advanced students, the results of the questionnaires shed light on the fact that almost all the students believed that the advantages of employing speaking portfolios exceed their disadvantages.One of the advantages concerns with self-reflection (self-assessment) was that students agreed that they were able to view their strengths and weaknesses by using speaking portfolios.Most of the students stated that the speaking portfolio had contributed to them to gain awareness of their language learning, in particular the improvement of speaking skill.This finding is congruent with Castaneda and Rodrigues Gonzalez' ( 2011) study.As in the present study, 80% of the participants reported the effectiveness of portfolio in terms of self-reflection and expressed their development in speaking performance.Another one deals with peer-feedback.In general, students discovered it useful to give peer-feedback and recognize their own mistakes by observing their peers' performances.The positive effect of peer-feedback seems to be dependent on individual differences including students' personal motivation.This implies that most of the students attempted to learn from their peers' performances and noticed their own flaws by evaluating their peers.This result is in parallel to what Anderson (1998) claimed about peer-feedback that is allowing students to get familiar with different perspectives and empowering their meta-cognitive skills.This also is in line with Wang and Chang's (2010) findings that revealed oral portfolios allowed students to learn not only from themselves, but also from their peers and their teachers.The following advantage of speaking portfolios indicate that students assessed speaking portfolio as an opportunity to practice speaking rather than as a method of assessment.The present results also uncover that students could employ a wide range of vocabulary, correct pronunciation and intonation via using speaking portfolios.This result is in agreement with what most of the studies has explored on different type of portfolios (Bolliger & Shepherd, 2010;Book, 1999;Castaneda & Rodriguez-Gonzalez, 2011;Danny-huang & Alan Hung, 2010;Wang & Chang, 2010).

Conclusion
By considering the obtained results of the current study, there is a significant difference between speaking video portfolios and improving the intermediate and advanced EFL students' speaking skill as students using video portfolios significantly progressed regarding speaking.The study also investigated the attitudes of students respecting learner autonomy, self-reflection, peer-feedback, practicality, motivation/anxiety and the improvement of speaking skills while using speaking video portfolios.The results indicated that the majority of the students thought that speaking portfolios had assisted their proficiency level in English specially in terms of self-reflection, peer-feedback and improvement of speaking skill.Furthermore, students believed that they could more productively make up for their mistakes and monitor their progress in speaking via speaking portfolios than they can in the traditional assessment.In addition, they were of the opinion that despite some anxiety and practicality problems, they desire to go on with such assessment format in the future.Lastly, although there were two students who totally rejected the employment of speaking portfolio, the overall findings show that this kind of alternative assessment was approved and appreciated by students for its outstanding benefits.

Implications
The findings of the present study may offer the following pedagogical implications for the instructors, administrators and test developers.One of the major finding is that speaking portfolio was discovered beneficial for overall development of speaking skill since it seems that students did not consider portfolio as a kind of testing, but rather as an opportunity to practice English.Therefore, such an alternative assessment could be recommended to foster speaking skill, although it should bear in mind that a careful, systematic, and well-organized preparation is necessary.
Another significant implication of the speaking portfolio implementation refers to the improvement in selfassessment skills of the students, which also leads up to learner autonomy as students need to take more responsibility (kılıc, 2009).As concluded in the finding of the present study, speaking portfolios could be applied to improve students' reflection skills.However, students should be instructed a bit longer before they get assessed which could be possible via video-recording carried out from time to time in classroom.
Yet another specific implication derived from the study is related to peer-feedback.According to the results of the study, students were encouraged to assess their peers' speaking performances and mentioned that they had learnt a lot from their peers.But it is important that instructors model suited behavior concerning sensitivity and respect for the interaction of students.Moreover, they should set criteria for peer evaluation by explaining that in what ways it could be done (Johnson & Rose, 1997).
Another implication about the use of speaking portfolios is associated with video-taping.Although, it required teachers with great deal of effort, most of the students seemed to be satisfied with watching their performances later and being capable of reflecting back on them.Interestingly, the students did not seem to look on this assessment as a kind of testing, but rather they seemed to be less anxious and more motivated to speak in classes.
Instructors who wish to decrease anxiety and increase motivation in their EFL classes can make use of such portfolios rather than traditional assessment.
To sum up, the speaking portfolio could greatly contribute to EFL students, instructors, and administrators to teach and assess oral skills as speaking is one of the critical achievement of what a student has learnt and how much she/he could creatively employ the language components such as grammar, vocabulary, and pronunciation (Khamkhien, 2010).

Limitations
Each study has its own limitations and may suffer some short comings.This study is not an exception.Here are some limitations of the study.The first limitation is that the study had to be completed in a limited time (almost 3 months, one term) as the researcher was permitted to carry out the study during this time thus in a further research longitudinal case studies to explore long-term effects of speaking portfolios could be done.Longitudinal studies can uncover the influence of speaking portfolios on students' proficiency levels and general academic implementation in EFL classrooms.The second one is that the instructors were not interviewed regarding the speaking portfolio implementation as a result the collected data were not able to examine instructors' attitudes therefore in order to complement the finding of this study, an interview could be administered with instructors to explore their attitudes towards the speaking portfolio implementation.

Table 1 .
Results of the ANOVA on the pretest

Table 2 .
Descriptive statistics of the posttest

Table 3 .
Result of the ANOVA on the posttest

Table 4 .
Results of multiple comparison using the Scheffe Test

Table 5 .
Frequency, percentage, and chi-square of students' questionnaires