Should Dictionaries be Used in Translation Tests and Examinations?

Motivated by the conflicting views regarding the use of the dictionary in translation tests and examinations this study was intended to verify the dictionary-free vs dictionary-based translation hypotheses. The subjects were 135 Arabic-speaking male and female EFL third-year university students. Agroup consisting of 62 students translated a text from English to Arabic without a dictionary at the beginning of the semester and translated the same text with a dictionary at the end of the semester. Another group of 73 students translated a text from Arabic to English twice in the same way in the same semester. Both groups used electronic mobile dictionaries in the second translation. The lexical errors were detected and statistically analyzed. The t-tests revealed a highly significant difference in favor of dictionary-based translation. The errors committed in the dictionary-based translation were remarkably less than those committed in dictionary-free translation. Further research is needed to settle the dispute.


Introduction
A dictionary -monolingual or bilingual -is an important source of language learning.Apart from the direct meaning and shades of meaning of a word, a dictionary may contain information about the usage of a word, together with its spelling and pronunciation, (McAlpine & Myles, 2003).Hamouda (2013, p. 227) adds that a dictionary is "a valuable reference to the language in situations where teachers are not available outside the classroom".According to Jin and Deifell (2013, p. 515) dictionaries "scaffold language learning both within and outside the classroom."Language learners usually use dictionaries for both comprehension and production and in so doing they may learn some words either intentionally or incidentally, (De Ridder, 2002;Pulido, 2007).
Another area where dictionaries are used is translation.Researchers (e.g.Hatherall, 1984;Jelveh & Nejadansari, 2013;Kamos, 2005;Starren & Thelen, 1990) agree that a dictionary is the translator's tool of the trade.A translator may consult a dictionary to comprehend the source text, to produce the target text and to verify his knowledge of the meaning or usage of a word.Like a language learner, a translator may learn some words incidentally or intentionally.Rogers and Ahmed (1998, p. 193) pointed out that there is a "love-hate relationship" between the translator and the dictionary.In some cases, it provides the translator with the meaning or equivalent he is looking for or helps him verify or confirm hypotheses.In other cases, it disappoints him due to inadequate coverage.Some other variables involved in the use of a dictionary include the user, the level of the difficulty of the task, and the type of the dictionary (monlingual/bilingual, printed/electronic), (Hulstijn & Atikins, 1998;Prichard, 2008).
This second purpose for the use of the dictionary is the topic of this article.It focuses on the use of the dictionary by EFL students in translation tests and examinations.This study gained impetus from the conflicting views regarding the use of the dictionary in translation tests and examinations.As we shall see in the following sections, researchers (e.g.Gorgis & Kharabsheh, 2009) and translation instructors are divided.Some of them believe that all types of dictionaries should be banned in translation tests and examinations while others allow them.The former think that translation students majoring in the foreign language (e.g.English) should be linguistically competent enough to translate without a dictionary and that the use of the dictionary is time consuming in test situations where the students have to complete the task in a specific period of time.The latter group believes that the use of the dictionary is realistic and reflects what the students will be doing in their future work life as translators.They also believe that the use of the dictionary involves problem-solving and decision-making skills that need to be taught and tested.They think that banning dictionaries will turn the task into a vocabulary test, not a translation test.Hence, the present researcher seeks to investigate this controversial issue by means of an experimental study.To the best of the knowledge of the researcher, very few systematic studies have so far been conducted in this area, especially in the Arab world.Rangelova and Echeandia (2003) administered a questionnaire to 26 students of English as a second language (ESL) to compare the dictionary with the mental lexicon.They pointed out that the dictionary is limited compared to the mental lexicon.However, the researchers did not test their hypothesis in relation to translation.Their belief that the way in which entries are presented in the dictionary does not reflect the link between the words in the human mind is more related to language learning rather than translation.In case of translation, the students need to link words with their meanings, not with other words.The dictionary may give other meanings of a word that are not stored in the mental lexicon.The researchers also relied on the students' perceptions rather than on what they actually do.

Related Studies
A more relevant study was conducted by Gorgis and Kharabsheh (2009).The first author (D.Gorgis) advocates 'dictionary-free' translation in tests and examinations whereas the second author (A.Kharabsheh) supports 'dictionary-based' translation.The former believes that the students who use dictionaries during training "must be linguistically and psychologically ready to take an examination without using a dictionary," (p.22).She thinks that "recourse to a dictionary is unavoidable at any stage of training including examinations," (p.22).Thus, the two researchers embarked on their study to see whether dictionaries should be used in tests and examinations or not.They asked two groups of Arabic-speaking EFL university students to translate some Arabic collocations into English in a test situation.The collocations were used in sentences.The dictionary-based test was given to 31 students in 2005.The students were allowed to use any type of dictionary available.The dictionary-free test was given to 40 students in 2008.The researchers concluded that the dictionary-based group outperformed the dictionary-free group.This finding "runs counter to the first author's speculation," (p.28); it also conflicts with Rangelova and Echeandia's (2003) findings.Gorgis and Kharabsheh's (2009) pioneering study constitutes an important step that can lead to a series of systematic attempts to settle this dictionary-based/dictionary-free translation argument in general and in in the Arab world in particular.However, their study suffers from some limitations that should be taken into account before the findings can be generalized.The tests focused on the translation of only one linguistic element (i.e.collocations) given in decontextualized sentences.Even within the area of collocations, the researchers focused on three lexical items only.They admit that not all Arabic collocations can be rendered into corresponding English collocations.Their reasons for focusing only on collocations -ease of assessment and ease of processing the scores -are not convincing.Although the researchers defined collocations as "non-idiomatic expressions" consisting mainly of two lexical items, four out of the ten examples they listed under the verb 'daraba' (= hit) were idiomatic expressions consisting of three or more lexical items (e.g.daraba biyad min hadeed = hit with an iron hand).Some expressions were not collocations (e.g.daraba Zayd Amr = Zayd hit Amr).
One of the arguments given for not allowing the dictionary in tests was that the students "have already had 15 years of exposure to English vocabulary."(p.23).However, these 15 years were academic years where the students studied English as a foreign language for a maximum of four hours per week as a school subject.From the present writer's experience in teaching EFL and translation at the university level in the Arab world, many students attain only an intermediate level or lower in classroom learning of EFL.Some of them fail to produce even a simple sentence.The two researchers (Gorgis and Kharabsheh) themselves admit that only top students opt to translate without a dictionary.
Gorgis and Kharabsheh's study lacks proper control because they collected the data from two different groups with a three-year gap between them.In addition, some of the Arabic collocations given to the students for translation were not accurate.For example, the verb 'hasara' (= narrow down, restrict, confine) cannot collocate with the Arabic equivalents of 'in a room, breathe, urine'.The proper Arabic verb would be 'habasa' (= lock up, hold, retain).Thus, the researchers did not test the translation process from Arabic to English and from English to Arabic in a realistic and natural manner by means of a properly controlled experiment.

Data Collection and Analysis
Motivated by the dispute over the use of dictionaries in translation tests and examinations and taking note of the drawbacks inherent in the previous studies reviewed above, the present study was intended to be a more rigorous attempt to resolve the dictionary-based/dictionary-free translation conflict.The subjects of the study were 135 Arabic-speaking male and female third year university students majoring in EFL.They were taking an English-Arabic-English translation course as one of the BA degree requirements.One group consisting of 62 students translated a text from English to Arabic without a dictionary at the beginning of the semester as part of the continuous assessment procedure.The same group translated the same text with a dictionary at the end of the semester.Another group consisting of 73 students from the same batch translated a text from Arabic to English twice: once without a dictionary at the beginning of the same semester and again with a dictionary at the end of the semester, (see Appendices A and B for the two texts).According to the procedure explained in the translation course description regarding continuous assessment tests, the students were given one hour to translate each of the two texts.All students completed the task within the specified time allowed.The two translations of the same text constituted 20% of the final score of the course.The students who did not take any one of the two versions of the two tests were excluded.In the second version of the two tests, all students used bilingual English-Arabic and Arabic-English mobile dictionaries because all of them had smart mobile phones and all of them were digital natives and adept at using the various features and applications of their devices.
Researchers agree that mobile phones are very common, fast, easy to use, small in size, can be customized, and helpful in tests, (Al-Jarf, 2001;Boonmoh, 2012;Jin & Deifell, 2013;Macintosh, 1998;Prichard, 2008).Jelveh and Nejadansari (2013, p. 35) add that the students have "famous dictionaries like Merriam Webster, Oxford, Cambridge, Longman, Collins and many more … in their mobile phones."Of course, the students can also download comprehensive bilingual English-Arabic and Arabic-English dictionaries such as Al-Mawrid.
Two bilingual university translation instructors cross-marked the translations focusing only on spelling and lexical errors for the purpose of the study.Grammar errors were ignored on the assumption that not all the students had sophisticated mobile dictionaries containing morphological and syntactic information.The spelling errors were not included in the statistical analysis because they were very few in both versions of the target texts.Thus, only the lexical errors were statistically analyzed.An error count was conducted to see if there was any difference between dictionary-based and dictionary-free translation.The following tables of descriptive statistics show the number, percentages and means of the lexical errors in the dictionary-free and dictionary-based translation of the two texts.Total 883 Table 1 shows that the 62 students committed 439 lexical errors in their dictionary-free English-Arabic translation (i.e.71.73% of the total number of errors made in the two versions).The dictionary-based errors were 173 (i.e.28.27).Table 2 shows that 883 lexical errors were made by 73 students in the two versions of the Arabic-English translation.The errors committed in the dictionary-free translation were 542 (61.38%) while the dictionary-based errors were 341 (38.62%).The two tables show that the lexical errors in the dictionary-free translation were more than those made in the dictionary-based translation.
Paired and independent group t-tests were used to see if the differences between the means of the groups were significant.In Table 3, the difference between the means of the lexical errors of the paired groups in the dictionary-free English-Arabic translation (7.08) and the dictionary-based translation (2.79) was highly significant (t = 13.853,df = 61, p .001).Similarly, Table 4 shows a highly significant difference between the means of the errors made in dictionary-free translation (7.42) and dictionary-based translation (4.68), (t = 10.062,df = 72, p .001).Table 5 shows that there was no significant difference between the means of the independent groups (7.08 and 7.41) in dictionary-free English-Arabic and Arabic-English translation, (t = 0.666, df = 133, p 0.506).The two groups committed almost the same number of lexical errors when they did not use the dictionary.Table 6 shows that the errors of the two groups decreased when they used the dictionary (2.79 and 4.68).However, the t-test result shows a significant difference between the means (t = 4.745, df = 133, p .001).The errors made in dictionary-based Arabic-English translation were significantly more than those made in dictionary-based English-Arabic translation.This could be attributed to the low proficiency level of some students in EFL which makes translation from the first language (Arabic) to the foreign language (English) more difficult than English to Arabic translation.In other words, EFL comprehension is easier than its production.It could also be due to the habit of picking the first EFL equivalent listed in the dictionary without verification.Informal discussions with some students revealed that they did not change some of the English words in the second translation assuming that they were correct.The significant difference between means of the two dictionary-based translations constitutes further evidence that students' proficiency level in EFL cannot be taken for granted.Dictionaries should not be banned on the grounds that the students are competent enough to translate without a dictionary.The argument in favor of the dictionary-free translation needs to be reconsidered.
From the foregoing descriptive and inferential statistical analyses, it is evident that the use of the dictionary improved the students'performance. Their performance in the dictionary-based translation was better than that of their dictionary-free translation with regard to the use of lexical items.It goes without saying that some degree of improvement in the second translation from English to Arabic and from Arabic to English could be due to possible language development through exposure to EFL during the semester.However, we can confidently attribute the improvement in the second translation to the use the dictionary since there was a highly significant difference between the means of the paired groups.

Summary and Conclusion
This study gained impetus from the dispute over the use of the dictionary in translation tests and examinations.
The supporters of the dictionary-free translation believe that the use of the dictionary is time consuming and that the university students should be linguistically competent enough to dispense with the dictionary.Researchers and translation instructors who advocate the use of the dictionary argue that a translation test is different from a language test and that the assumption of students' linguistic competence in the languages involved is an ambitious speculation.They also believe that the use of the dictionary is a skill that should be taught and tested.
The small number of studies conducted in this area reported conflicting findings.Hence, it was the purpose of this study to verify such findings taking into account the limitations inherent in the previous studies.Data for this study were collected from two groups of Arabic-speaking EFL university students.One group was asked to translate a text from English into Arabic without a dictionary at the beginning of the semester and the same text again with a dictionary at the end of the semester.Another group translated a text from Arabic to English in the same way.The lexical errors were detected and statistically analyzed.The results revealed a significant difference between the means of the dictionary-free and dictionary-based translations.The use of the dictionary improved the students' translation, a finding that is in line with the hopes and aspirations of the translators in their work life.
This study lends support to those who advocate the use of the dictionary in tests and examinations.The supporters of dictionary-free translation need to note that the students' proficiency level may help if they read a text only for general comprehension; they can ignore some words and guess the meanings of some others.In case of translation, however, the students need to find the equivalent of each unfamiliar word in the text; they also need to verify their knowledge of other words that they are not sure of.The argument that using the dictionary consumes time does not hold water with the use of electronic mobile dictionaries.When the dictionary is banned, the students may be deprived of the possibility of learning some words incidentally or intentionally in their quest for meanings or equivalents.Needless to say, more studies are need to further verify the dictionary-free vs dictionary-based translation hypotheses with more students at different levels of proficiency and with different languages.Due to practical course and time constraints the present researcher could not have all of the 135 students translate the two different texts (English-Arabic and Arabic-English) twice (with a dictionary and without a dictionary) to conduct a paired group t-test for all of the subjects.Such an arrangement might have led to a more reliable conclusion regarding the use or non-use of the dictionary.Since this study focused only on the lexical errors, further studies can include grammar and spelling errors.The dictionary variable (electronic vs paper) could also be manipulated.Random sampling may also be considered, if possible, instead of the intact group design used in this study.

Table 1 .
Number, percentages and means of errors in dictionary-free and dictionary-based English-Arabic translation

Table 2 .
Number, percentages and means of errors in dictionary-free and dictionary-based Arabic-English translation

Table 3 .
Paired group comparison of the means of the errors in the English-Arabic translation

Table 4 .
Independent group comparison of the means of the errors in the Arabic-English translation

Table 5 .
Comparison of means of independent groups in dictionary-free translation

Table 6 .
Comparison of means of independent groups in dictionary-based translation