Review on China Accreditation Test for Translators and Interpreters ( CATTI )

Since its first pilot study was launched in 2003, China Accreditation Test for Translators and Interpreters (CATTI) has developed into the most authoritative translation and interpretation proficiency qualification accreditation test in China and played an important role in assessing and cultivating translaters and interpreters. Based on the analysis of its form and content, the current study appraises CATTI in terms of reliability, validity, practicality, fairness and washback. The results suggest that CATTI has a satisfying validity and reliability, but the amount and types of test items, test content, evaluation rubrics and transparency are expected to be improved.


Test Purpose
The purpose of the CATTI is to measure and accredit the proficiency and competence of translators and interpreters nationwide, and to strengthen the construction of foreign language translation talents in China.The CATTI consists of two types of tests: translation and interpretation (consecutive interpretation and simultaneous interpretation) of seven languages, namely, English, Japanese, French, Arabic, Russian, German and Spanish.It includes four levels: Senior Translator and Interpreter, Level 1 Translator and Interpreter, Level 2 Translator and Interpreter and Level 3 Translator and Interpreter.

Length and Administration
The CATTI is administered by the China Foreign Languages Publishing Administration (CFLPA) on behalf of the Ministry of Human Resources and Social Security, the People's Republic of China.Tests of English language of Level 2 and 3 are launched twice a year, while tests of the other six languages and tests of English language of Level 1 are launched once a year, either in May or in November.Since the CATTI includes two types of test of four levels, different tests take different time to complete.

Scores
For CATTI Level 2 and 3, test takers scoring 60 or above on both tests for Comprehensive Ability and Practice will be awarded the Translation and Interpretation Proficiency Qualification Certificate of the People's Republic of China.For CATTI Level 1, examinees have to score above 60 or above and achieve the agreement of experts before they obtain the certificate.For senior translators and interpreters, they need to meet the requirements of experts' panel and they are not included in the National test (Zhou & Yu, 2011).As a consequence, this review will focus on the test for Level 2 and Level 3 Translator and Interpreter in English.

Price
The prices for the CATTI vary in different tests held in different provinces of China and also change with time.The test center in each province can decide its own price on the basis of baseline fees required by Ministry of Finance and fees paid for the places and invigilators.

General Description
The CATTI is an accredited test accessible for every citizen of Chinese mainland，Hongkong, Macao, and Taiwan and foreigners working legally in China.The first pilot test of CATTI's English Level 2 and 3 was launched on Dec. 16 th , 2003 in three cities: Beijing, Shanghai, and Guangzhou.The number of test takers was 1,629.492 of them passed it and obtained the certificate.Since then, the scale of the CATTI has expanded gradually.Till 2009, the total amount of the test-takers who had applied for this test was more than 96,000, and over 14,300 test-takers passed the test.
The CATTI test has experienced some small revisions in history.In 2010, it slightly revised the score arrangement and the test content in the English Translation Comprehensive Ability test.The test content and format of the CATTI (Level 2 and 3 translation and interpretation test) are displayed in Table 1 (see Appendix).

Appraisal of the CATTI
The following appraisal is mainly based on the test qualities and practices in Standards for Educational and Psychological Testing (AERA, 1999).And we will focus on the validity, reliability, practicality, washback and impact of the CATTI.

Validity and Reliability
According to the Standards for Educational and Psychological Testing (AERA, 1999, p. 9), validity refers to "the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests".In 2005, CFLPA's National Translation Test and Appraisal Center set up a special research group to study the ways to maintain and improve consistency of the test paper for Level 2 and 3 of CATTI.Zhao, Wang, and Lu (2007) analyzed the standards, procedures, requirements and operational details in preparing of test papers.They reported that the CATTI for translators had high construct validity and could effectively assess translators' language competence and translation ability.Lu, Liu, and Gong (2007) pointed out that the design and construct of the CATTI for interpreters conformed to the testing theories and the examination syllabus, and had a satisfying content and construct validity after analyzing the test papers for Level 2 and Level 3 from 2003 to 2005.The above two studies were the starting point of validity study of the CATTI and provided suggestions for the CATTI developers.For example, the amount and types of test items and words beyond the syllabus should be better controlled and specified in Interpretation Test (Lu et al., 2007).
After 2007, another two validation studies were conducted to further evaluate the CATTI for translators.Li (2011) analyzed the five Translation Practice tests of Level 3 from 2006 to 2010, and concluded that this test had a high face validity, construct validity and content validity.Her study proved that the Translation Practice test was acceptable to most test takers and tested examinees' language ability and translation skills.At the same time, she suggested that the basic information of the source passages and a wider range of topics should be provided, and the arrangement of different genre can be more balanced.Zhou (2012) analyzed three test papers for Level 2 of CATTI for translators from 2010 to 2011, and found that the CATTI had satisfying internal validity, external validity and using validity.At the same time, she thought that the amount of test items and evaluation standards could be improved to enhance the validity.Wang (2011) systematically evaluated the oral speeches of CATTI from 2003 to 2010 and reconfirmed the high construct validity of Level 2 Consecutive Interpreting Test.
In addition to test validity, reliability is another important part of a good test.Reliability is "the consistency of [measurements] when the testing procedure is repeated on a population of individuals or groups" (AERA, 1999, p. 25) and explores "how much of an individual's test performance is due to measurement error" (Bachman, 1999, p. 161).Zhao, Wang, and Lu (2007) analyzed the examination syllabus and the reading materials of the CATTI translation tests from 2003 to 2005.They found that the amount of test items, important points and the difficulty level were consistent with the syllabus and the current situation of China.The consistency of tests assures the reliability from the perspective of test design and grading.However, the scoring of Translation/Interpretation Practice is subjective, and it is difficult to interpret its reliability from the viewpoint of scores.Even though the recordings of the test takers of CATTI will be judged and graded by two or more test reviewers, the scoring process cannot be elaborated in a quantitative way and influences the overall reliability of the test (Wang, 2011).
"Test validation is an ongoing process of demonstrating that a particular interpretation of a test score is justified" (Jin & Fan, 2011).By now, only a few researches have been done to study the validity and reliability of the Level 2 and Level 3 of the CATTI (English).Although these studies have proved the validity and reliability of CATTI from different perspectives, and provided suggestions for the test developers to improve the test, they only involve logical analysis, and lack data analysis of real test scores to rationalize the research.Therefore, more researches, especially empirical researches on estimating how different factors influence the test design and test scores, are urgently needed to validate the CATTI.

Practicality and Fairness
As is presented in the description part, it can be seen that the number of CATTI test takers per test proliferated nearly ten times compared with that ten years ago.The semi-directed oral test format and computer-assisted recording system adopted by the interpretation tests can guarantee the efficiency and practicality.In the Interpretation Practice tests, 30 test takers in the same test room sit separately so that they neither disturb nor be disturbed by others.During the test, test takers have an ear-phone and sound recorder to listen to the recording and interpret to the recorder.After the test, their recordings are sent to the scorers confidentially.The scorers can listen to the recording of their answers more than once if necessary, which to some extent ensures the fairness and objectivity of the scoring.

Impact and Washback
Since its birth, the CATTI has drawn large attention.The Translators Association of China admits every test taker who has received the certificate to join in the association.A series of researches on translation and interpretation have been done to explore better teaching methods.Liu (2007) analyzed the CATTI interpretation test and gave some suggestions for the reform of interpretation education.Moreover, some Chinese higher education institutions have incorporated the test into curriculum and require postgraduates majoring in translation or interpretation to pass Level 2. Besides, some universities even begin to have their school-based curriculum according to CATTI.For example, Northwest Normal University began to develop a school-based curriculum for their translation major undergraduates (Dong, 2013).According to Dong (2013), the combination of translation education and the CATTI is shown in three aspects: 1) the combination of the curriculum objectives and the CATTI test objectives; 2) the combination of the teaching contents and the CATTI (Level 3) course books; 3) the combination of the teaching methods and evaluation rubrics with the CATTI testing format.Moreover, some other countries have also shown great interest in the test.It is reported that visitors to the CATTI website (www.catti.net.cn)come from over 50 countries and regions.Scholars from Japan, the Republic of Korea, Singapore and France have come to China to learn about the test and explore ways of cooperation in this field.

Conclusion
As a criterion-referenced test, the CATTI is designed and developed in accordance with the CATTI syllabus and the developing requirements of foreign language translators and interpreters in China.By now, the test has grown to be the most authoritative translation and interpretation proficiency qualification accreditation test and widely acknowledged by society.Validity and reliability of the CATTI of Level 2 and Level 3 has been supported by some studies through logical analysis and material analysis, but the amount and types of test items, test content, and evaluation rubrics are expected to be improved.It is, therefore, rather urgent for test developers to conduct more empirical validation researches to estimate how different factors affect the test design and test scores, and provide insightful suggestions for further enhancing the validity and reliability and improving practicality.Given the significant role the CATTI plays in evaluating the proficiency of translation and interpretation talents, the administrators should take the responsibility and spare more effort to make it more valid and transparent to test takers.
length is given in minutes; L 2/3 = Level 2 or Level 3; E-C = English to Chinese; C-E = Chinese to English; Wpm = words per minute.