A Leadership Assessment Center for Graduate Students: Case of Saudi Arabia

The Khaled Alfaisal Administrative leadership Development Program which was conducted in the early part of 2014 had provided a global exposure blended with experiential learning for select graduate university students in Saudi Arabia. The key role of the assessment centre was to observe, evaluate and eventually rate each participant’s performance. The assessment centres was conducted with the aim of creating new opportunities for Saudi graduate students by developing them as leader and make them more marketable. In the program, the students had undergone a series of activities under the observation of university faculties and outside consultants. The competencies examined were intricately related to leadership behaviour which was to be identified for the leadership program and future assignments. The paper also analysed the effectiveness of the assessment centre against the unique Saudi context and discussed the suggestions and future research implications.

fast social change to be in sync with economic developments.There is a constant struggle to balance between the roadmap to the future and coordinating the new insights, principles and values with the past standards.
Education and globalization have impacted the Arab world beyond measure both positively and negatively.Modernized facilities have enhanced and simplified day to day living in the Arab world however; the greatest of impact has been felt on the family unit and societal structure.Arab society struggles with the dominance of traditionalism over individuality, creativity and modernism (Churchmuch, 2006, p. 5).In this perspective, I would like to differentiate the Arab or rather Saudi practices in education, childhood experiences that should make their future assessments customized even if the assessment structure has been taken from the west.The educational conditioning fostered early by the memorization of the Quran leads to rote methods of learning which involves lack of application or practice with other facts.With regard to child-rearing, the style fluctuates between two extremes-authoritarianism and over-protection (Churchmuch, 2006).
"This hinders the individual's ability to develop critical thinking and problem solving skills.These practices all leave little room for the individual to question information and develop skills needed for discrimination of information which are practiced within westernized curriculums and globalized societies."(Churchmuch, 2006, p. 7).
The Saudi child's development of curiosity, sense of independence and leadership activities are unconsciously curbed.This is in sharp contrast to Western way of educating the child where the emphasis is on individualism, rationalism and analytical abilities.The Arab culture, which is a high context culture differs greatly with the Western one (high context), in the communication strategies too.One has to understand also that the collective context also dilutes the individual sense of responsibility and accountability where the onus lies on the group.There is a conflict between the select adoptions by the Arab world of the Western aspects thus isolating them from the background upon which they have been formulated."Arab society has adopted selected items of Western technology while rejecting the principles and values that brought it about.They have raised standards of living without combating economic disparities and expanded education while overlooking its relevance and utilization" (Barakat, 1993).Even the individual characteristics as well as conflict-resolution strategies that pose a barrier for the Saudi (Arab) ideologies have been sidelined abruptly.
Additionally there has been no past literature on the assessment center activities in Saudi Arabia, hence in a way this paper introduces the application of a student assessment method methodology with its strengths and shortcomings.
At the onset before proceeding to describe the steps and phases of the Assessment center, I will give a brief summary of the external consultant profile and the inception of the process.

Significance of Leadership Education in Saudi Arabia
Leadership is an important part of university students' education (Alexander, 2011).The development of leadership competencies is crucial for the sustained growth of a country.Hence it becomes imperative to guide, support and assist the young educated population of a nation to emerge as successful leaders in the various fields they choose.In recent years Saudi Arabia had embarked on the Saudisation program which seeks to gradually replace foreign workers with local ones.Ironically there lies a skill shortage among the Saudi young graduates and also the older segment of the population.Chaar (2010) had pointed out aptly that Saudi Arabia has been facing a shortage of local leaders in the private sector.He had addressed serious concerns to the variables causing growing leadership shortage, namely, bulging young population, disparity between growth and the available talent in the private sector, alarming rise of unemployment.Though the number of people involved in leadership programs on college and university campuses around the world continues to grow (Prince, 2001) but the fact remains that only academic courses in leadership cannot elicit the desired behavioural outcomes.It becomes imperative that such development programs should be conducted from time to time to ingrain valuable leadership competencies.
In most universities, the development of student leadership skills is not part of the academic curriculum but relegated to extracurricular activities (i.e., it is regarded as part of student's non-academic activities in clubs and organizations).Helping students develop the integrity and strength of character that prepare them for leadership may be one of the most challenging and important goals of higher education" (King, 1997, p. 87).Increasingly, higher education is being turned to as a source for potential change given its significant role in developing leadership capacity among today's youth (Astin, 1993;Astin, 2000;Morse, 1989Morse, , 2004)).As discussed above leadership concepts of the academic courses needs to be blended with the practical dimensions of experiential learning.Research indicates that students can and do increase their leadership skills during the college years (Pascarella & Terenzini, 2005) and that increases in leadership development in turn enhance the self-efficacy, civic engagement, character development, academic performance, and personal development of students (Benson & Saito, 2001;Fertman & Van Linden, 1999;Komives et al., 2005;Scales & Leffort, 1999;Sipe, Ma, & Gambone, 1998;Van & Fertman, 1998).

Background of the External Consultancy and the Domestic Sponsor
The Singapore Cooperation Enterprise (SCE) was established by the Ministry of Trade and Industry as well as the ministry of foreign affairs in 2006 (http://www.sce.gov.sg/about-us.aspx).
The special thrust of the enterprise had been to cater to the foreign clients through distinctive, individualized programs.The range of services to fulfil the foreign government's needs is either offered through study visits or training programs or a combination of both.The study visits consist of a couple of working days and include meetings, industrial visits and customized workshops.
The unique feature is that the agency has a number of competent experts who have the capability of examining the prevailing systems, do a gap analysis and suggest suitable models based on Singapore's internal experiences.The experts can also guide projects and support the government in partnership programs.A key area is project implementation which is usually done in phases in Singapore and the host country.
As SCE is an agency of the government so it has the right to sign a Memorandum of Understanding (MOU) and other agreements related to the foreign governments related to the foreign governments and their states.SCE in a way had been extensively collaborating with other countries to train and develop in a various public sectors.From the start, it had been working in over 30 countries and Middle -East region is significant strategic region.
Manarat Al-Oloom for Researches Ltd. was been the official sponsor of the Qiyadat program.They had engaged SCE to design and conduct the program.There was a lead consultant from SCE on this program.There were initial discussions, query handling after which an agreement between King Abdulaziz University (KAAU) and Manarat was forged.The principal consultant identified two other professionals who had the requisite expertise and experience in selection matters to design the Assessment center.After this process, there were internal assessors identified and trained for few days with mock assessment situations, presentations, analysis and feedback.These assessors were the senior faculty members of King Abdulaziz University.During the workshop, assessors were informed about how the competencies were identified and how the AC was designed as well as providing them with knowledge of the context in which the assessment was being conducted.The relevance and definition of each dimension was explained to the assessors.Assessor training is a crucial part of the AC technique (Joiner, 2000).In order to avoid errors, Caldwell, Thornton and Gruys (2003) suggested that assessors should receive training with regard to the process of observing and classifying dimensions and behaviours, as well as demonstrate skill in order to evaluate assesses' performance properly.
Assessment centers (ACs) are developed around different dimensions or competencies which are associated to a specific job or work context.Performance-related dimensions, such as decision-making, problem-solving, analytical thinking and adaptability, are examples of dimensions assessed and the various exercises included in the AC to measure these dimensions are referred to as methods (Greyling, Visser, & Fourie, 2003;Lievens, 1998).The AC is termed a multi-method, multitrait, multi-rater approach (various exercises are used and multiple dimensions are observed and measured by more than one assessor) and the assessors are normally trained in observing the dimensions exhibited and this method allows for the measurement of multiple competencies that are otherwise difficult to measure (Thornton & Rupp, 2006).

Method
The AC methodology under study had encompassed competency dimensions, the exercises and the training of assessors.In the process, behavioural anchors were assigned to each competency dimension and rated on a 5-point scale, namely, very weak, weak, acceptable, strong and very strong.
There was a Pre-Assessment Centre phase which consisted of the initial screening based on two important aspects, namely qualifications and leadership experience.This was done by a basic online English and ability test screening.It filtered out a huge crowd of unsuitable candidates.There were a total of 484 applications; shortlisting was based on English proficiency and other criteria like leadership experience and awards.On 12th November, 2013 the applications were closed.253 students were then called for the Ability test but 221 showed up.By 10th December, 2013 online ability test screening was done.Between 9-13th February, 128 students were called for Assessment center but 115 showed up.43 students turned up for the Structured Interview (SI) between 23rd to 26th Feb. Post SI there was a discussion with the sponsor Manarat (27th Feb) and finally 34 students were selected for the program.
Seven key competencies were identified and assessed in multiple exercises consisting of the group and individual presentations (8 candidates in a group) based on a detailed cross -functional case with department specific information.
According to Burnett and Dutsch (2006) competency is defined as the individual's work-related characteristics such as skills, knowledge, attitudes, beliefs, motives and traits.Similarly, Heinsman et al. (2007) defined competency as a cluster of skills, knowledge, abilities or personality characteristics.According to Mirabile (1997), competency refers to knowledge, skill, ability or characteristics that are closely related to high performance in a specific job.Spencer and Spencer (1993) identified competency as the individual's underlying internal characteristics that result in effective and superior performance.Spencer and Spencer (1993) suggest that competency is composed of motive, traits, self-concept and knowledge and skills.At the top of the iceberg are knowledge and skills, which are visible elements of competence.Both of these elements are surface competencies, which are relatively easy to develop (Jucevicience & Lepaite, 2005).The iceberg model consists of five types of competency characteristics, i.e. motive, traits, self-concept, knowledge and skills.These are defined in the various researches (Vathanophas & Thai-Ngam, 2007;Monk, 2001).
The main competencies which they wanted to identify among the Saudi graduate students were intellectual capacity, intellectual curiosity, motivation to lead, interest to contribute to society, basic interpersonal skills, character and integrity and English communication Skills.
Followed by the group presentation structured (SI) and detailed interviews were taken and finally the individuals (at this phase lesser in number) were accepted for the Leadership Development program.

Pre-Assessment and SI
Once the initial screening was over there was a workshop for the assessors who would be conducting the structured interviews for the shortlisted Saudi young students.The salient objectives of the workshop:  To hone skills in conducting Structured Interviews (SI)  To evaluate candidates using objective and systematic approach.


To practice SI interview questions, probes and scoring.
The Assessors were the Senior Faculty members of KAAU along with the external assessment team of SCE .The Assessment consisted mainly of the SI process which was after the individual and group discussions.
The SI had the following phases: Introduction of Interview, Opening Background Questions, Introducing the Competency/ Topic, Probing for specific past examples.
A major finding in interview research a few years ago was that interviewer judgments based on structured interviews were more predictive of job performance than those from unstructured interviews (Macan, 2009).Schmidt and Zimmerman (2004) explored a measurement-error explanation.They hypothesized that structured interviews show higher validity for predicting job performance compared to unstructured interviews because structured interview ratings are more reliable.
The introduction section had the welcoming note with use of ice-breakers.The selection process was explained in a positive and concise manner, however, the interviewee had to be explained that due to the structured nature of the interview, they had to provide specific examples from the past on three main areas; -How they have learnt new things?
-How they have helped others?
-How they have worked with others that have different beliefs?
It was also made clear to them that the assessors would be taking notes and may interrupt the candidate to monitor the key areas to be assessed.The second phase consisted of "Opening Background Questions".The main purpose of these questions was to break the ice and to understand the interviewee's general experience and competencies.Some examples of the questions asked were: 1) What prompted you to apply to the leadership program?
2) What do you want to achieve?
3) Tell me what you enjoy most at the university 4) What were some current challenges?
The third section comprised of the competency/Topic where the actual pre-decided competencies were assessed.
Intellectual curiosity: It refers to the openness and curiosity to learn new things and challenge the status quo.
The desired behavior which were to be observed were the student's inquisitiveness to learn new knowledge and skills, demonstrates a range of interests across different disciplines, shows interest to acquire in-depth understanding of issues, experiments with new ideas to try out fresh approaches.
The topic was introduced first through the questions cited below: "How often do you learn new things that are outside the normal school curriculum?" The probing questions followed: -"When was that?How long did it take?" -"How did you go about learning it?How did you deepen understanding?How did you find time?"offering concrete assistance to the underprivileged in the community, making appropriate personal sacrifices for the benefit of others in the society.Some of the opening questions were; -"How much interest do you have in community work, why?" -"How often do you engage in such activities to help those that are in need ?" As the interview progressed there were in-depth questions like: -Give an example of a time when you have helped a group or community in a significant way.
-What was your role?Who else were involved?When was that?
-How did you contribute?What did you do to help?How did you feel?The session started with the question: -"Lastly, we would like to find out more about your personal values and beliefs".
There were more extensive questions: -"Tell us a time when you encountered someone, who has acted dishonestly or unprofessionally in your view" -"What did you do?What was your intention?What did you learn?
Finally, the last phase, namely, the interview closing was done.The assessors marked the positive and negative competencies of each candidate.

Discussion & Limitation
Apart from the Assessment Centre (AC) scores, the students were banded by their performance.From the results obtained it was found that intellectual abilities demonstrated a significant correlation.The other variables did not show significant relationship, which may have been due to a restriction of range since students who had secured a certain AC score were accepted.This move has been appreciable because it reduced the level of criterion-contamination which again has been corroborated by previous research (Klimoski & Brickner, 1987) which points attention to the evidence that supports the possibility that criterion-contamination has caused inflated performance scores of those who succeed during ACs and deflated performance in those who do not succeed during ACs.
According to Lievens (2002) ACs are advantageous in that participants receive detailed feedback concerning their strengths and weaknesses on the dimensions measured.However, there was a different stance taken in this assessment center program, the feedback which is an integral aspect was not given to the students unlike the job candidates/employees. Finally there was the intense meeting and discussion between the external and internal assessors to round-up scores, re-examine candidate's performance through joint discussions with an effort to complement each other's assessments.Heindl (1997) and Thornton & Rupp (2006) have corroborated this aspect in their respective studies where they had stressed on the integration of the assessment obtained from different Assessors and different methods, usually during the wash-up session.
The Assessment Centre had some shortcomings.Firstly, the assessment flow involved ratings in the different categories separately based upon some pre-defined competencies.In my view it is a serious limitation because it obstructs a broader and comprehensive evaluation by the assessors.Schema-driven training would have been more beneficial in this context rather than the traditional assessment.Past research indicates that especially schema-driven training might be beneficial in terms of increasing inter-rater reliability, dimension differentiation, differential accuracy, and even criterion-related validity (Goodstone & Lopez, 2001;Lievens, 2001a;Schleicher et al., 2002).Schema-driven training (frame-of-reference training) teaches raters to use a specific performance theory as a mental scheme to 'scan' the behavioral stream for relevant incidents and to place these incidents -as they are observed-in performance categories.Such a training seems to be a useful complement to the traditional data-driven training that teaches assessors to strictly distinguish various rating phases (observation, classification, and evaluation) and to proceed to another phase, only when the previous one is finished (Lievens & Thornton, 2005).Essentially, raters have their own higher-order factor that they use to define behaviors (Hawthorne, 2011).Additionally, there are certain personality dimensions that influence raters to be more "tender-hearted" which may cause them to introduce excessive leniency into their overall ratings of assesses (Bartels & Doverspike, 1997).
Secondly, the proportion of senior faculty members used for the AC was more than the expert psychologists.More so, these faculty members were a diverse mix of Saudi and expatriate workforce with differences in socio-cultural perceptions and hence at times consensus reaching process was challenging.Mr Gavin Tan, one of the external assessors had rightly remarked: "typical of many assessors who were new, they sometimes had difficulties placing the observed behaviours in the right competency (criteria).Or they double penalized or credited the same behaviours in more than one criterion.Past research shows that the use of psychologists as assessors improved the construct validity (Gaugler et al., 1987;Jackson, Stillman, & Atkins, 2005;Lievens, 1998;Lievens & Klimoski, 2001).Additionally, the rating approach used may have influenced convergent and discriminant validity.
Thirdly, the assessors were given to rate all the dimensions and exercises related to them.This may have increased the cognitive loading that an assessor has to deal with while rating an exercise.There is evidence that it is important to have assessors only rate one exercise, and ideally only one dimension within an exercise, as it cuts down on common rater variance (Kolk, Born, & Van der Flier, 2002).
Some of these problems can be alleviated with how this stage of the process is addressed in training, and with the tools that are given to assessors (Thornton et al., 2006).Tools and training should direct assessors to focus on their task as behavior recorders and move them away from making inferences about behavior or letting their personal biases influence ratings.
Additionally, people have been shown to make better decisions and be better able to be self-critical when their accountability in the situation is made salient to them (Tetlock, 1983).It may be useful to consult literature on team design and processes for tips on how to best train teams of assessors to work together and address some of the above concerns with positive group dynamics (Jex & Britt, 2008).
Lastly, despite the rigorous training of the academic assessors prior to the AC program, there may have been some gaps and loopholes in the process due to the paucity of time.Research has found that assessors who received insufficient training to perform assessor duties were more likely to provide inaccurate ratings and ratings that were influenced by personal biases (Spychalski et al., 1997).
In future, the AC should design and implement ACs that is customized according to the Saudi psyche and culture.The unique cultural perspective and attributes of the Saudi Arabian context should be blended with the constructs of the evaluation instruments.
As mentioned before, the AC designers should not incorporate a large number of competency dimensions that seek to confuse the assessors.Future research can reflect the effectiveness of using a small number of dimensions (that are clearly defined and unrelated) for final selection phases.Having only a limited number of differentiated dimensions can make it convenient for the assessment process.The above recommendations will assist in reducing the cognitive demands that assessors experience during observation and classifying of behaviours demonstrated by the candidates (Nako, 2011).Thornton (1992) suggests the use of no more than seven dimensions for the assessment processes.

Future Implications
Future research should clearly focus on the effectiveness of such assessment centers in selecting Saudi leaders through longitudinal studies on faculty perception, external consultant reviews and above all the successful placement of the young graduates in different fields post development programs.These assessments done in a sophisticated and professional manner can gradually minimize the complaints made by educational researchers and private sector employers about the mismatch between labour force needs and the nature of educational qualifications, such as Saudi youth's lack of 'soft skills' and work ethics in the near future.
There have been a host of leadership programs being conducted in recent times to identify potential talent; however a lot remains to be done in terms of proper assessment and evaluation of the key characteristics.
Students spend a majority of their academic time in the colleges and institutions so the gradual, consistent leadership development and assessment can be fostered in the college/university through academic and non-academic exercises by the faculty members.There can be future research on the other students who were not selected, the competencies lacking and incorporating the areas lacking with further development by the college or university, followed by assessments after the development period.In this manner the effectiveness of the programs done can also be evaluated from time to time.
Character and Integrity: This dimension demonstrates high ethical standards and integrity and has the moral courage to stand for what is right.The desired behavior indicates strong personal beliefs and ethical values, identifies and points out undesirable or unethical behavior, does what is right despite opposition or social pressure, taking ownership of mistakes made and learns from mistakes.

Table 1 .
Assessment criteria by stages

The Process Flow of the Structured Interview
-"How has this learning helped you?To what extent did you apply what you have learnt?"Interest to contribute to society: This refers to the capacity to think beyond self and undertake efforts for the benefit of others.The expected behavior in this area is related to empathetic gesture towards less fortunate, Occasionally they may have missed out what the students failed to do.It was easy to see what students did, harder to notice what the students did not do and could have done."