Survey on Gesture Recognition for Hand Image Postures

One of the attractive methods for providing natural human-computer interaction is the use of the hand as an input device rather than the cumbersome devices such as keyboards and mice, which need the user to be located in a specific location to use these devices. Since human hand is an articulated object, it is an open issue to discuss. The most important thing in hand gesture recognition system is the input features, and the selection of good features representation. This paper presents a review study on the hand postures and gesture recognition methods, which is considered to be a challenging problem in the human-computer interaction context and promising as well. Many applications and techniques were discussed here with the explanation of system recognition framework and its main phases


Introduction
Gestures considered as a natural way of communication among human, since it is a physical movement of hands, arms, or body which conveying meaningful information.Gesture recognition then, is the interpretation of that movement as semantically meanings command.Gesture recognition has been studied in widely topics, and has a wide range of applications such as recognizing of sign language, human computer interaction (HCI), robot control, machine vision, smart surveillance, visual environments manipulating, etc (Ibraheem, 2011).
Although the term hand gesture and hand posture refers to the same meaning but there are some differences between them.Hand posture can be defined as the static movement, holding the hand with specific pose is a posture, for example a victory sign, pointing, and thumbs up.While in a complex posture, the fingers could be bent at any angle.The gesture can be defined as a dynamic movement, such as waving good-bye (Pavlovic, 1997;Mitra, 2007).A complex gesture is one that includes three things as mentioned by Mitra (2007) which are finger movement, wrist movement and changes in the hand's position and orientation.Examples of this type of gesture are the American Sign Language signs (Pavlovic, 1997).
Present approaches can be divided into Vision Based approaches and Data-Glove approaches.Data-Glove based approaches need a cumbersome devices have to be wore by user which connect the device to the computer, and that would reduce the natural level of the user's interaction with the computer (Mitra, 2007), for more details about data glove approaches, a survey on data glove are available in (Dipietro, 2008).Vision-based approaches, deals with some properties such as texture and color (Mitra, 2007) for a gesture analyzing; while tracking devices cannot.The number of cameras used by those techniques can be different; the speed and latency; and the environment structure, such as lighting condition or movement speed restrictions (Mitra, 2007).Examples of vision-based approaches are mentioned in (Zhu, 1998;Porta, 2002).Figure 1 shows an example of vision based and data glove based system.(Lamberti, ic (1997) (Murthy, 2009;Hasan, 2011a).

Min (1 psycho
C. Low Level Features based Approaches: Some gesture applications required a mapping between the input video and the gesture.Many researchers considered the full hand reconstruction is not essential for gesture system (Murthy, 2009;Hasan, 2011a).An example of low level features is, some geometric features which can extracted quickly and considered robust to noise (Murthy, 2009).Low-level features include: the centroid of the hand region (Pavlovic, 1997), an elliptical bounding region of the hand, edges, regions (Pavlovic, 1997), silhouettes (Erol, 2007), moments, and histograms (Pavlovic, 1997;Bilal, 2011).

Application Areas for Hand Postures and Gestures
Various applications have been used for hand postures and gestures as alternative level of interaction in different application domains, as mentioned in (Mitra, 2007): including virtual environments, smart surveillance, sign language translation, medical systems etc.This section gives a brief overview of some gesture recognition application areas (Mitra, 2007;Joseph, 1999).Hardware decreasing and reducing the processing cost can play a major factor for making the gesture recognition system the future setting and can offer more practical areas as well (Garg, 2009).Table 2 demonstrates some applications on gesture recognition system.

Sign Language Recognition
Sign language considered as an important and interesting application fields of hand posture and gesture recognition system (Joseph, 1999), where many systems have been applied for this purpose (Joseph, 1999).Sign language has special importance for communications since the gestures are the way used for interpretation and explanation of specific subject (Murthy, 2009).It can be used for disabled peopled when communicating with the other people, and with the computer as well (Murthy, 2009).American Sign Language in (Vogler, 2001;Starner, 2002) is one example that has received significant attention in the gesture literature.Kim (1996) recognized Korean Sign Language (KSL).Cho (2006) define a new gesture recognition algorithm for Korean scripts.Liang (1998) introduced lexicon of 250 vocabularies in Taiwanese Sign Language (TWL), Maraqa (2008) recognized Arabic Sign Language (ArSL).Murakami (1999) recognized Japanese sign language alphabets and words, they could recognize 42 alphabets and 10 words using two types of Neural Networks.

Robotics, Human Manipulation and Instruction
One of the effective applications that can utilize hand postures and gestures is robot tele-manipulation (Joseph, 1999).Telerobotic applications are typically classified under space exploration and military research domain (Murthy, 2009).Using gestures for controlling robots is corresponding to virtual reality interaction system (Murthy, 2009).Recent researches used postures and gestures to learn the robot some interaction commands by explaining its appropriate meaning for the robot as an action (Joseph, 1999).Various researches for robot control applications are implemented in (Malima, 2006;Wang, 2003;Bertsch, 2009).

Virtual Reality
For virtual reality application gestures have considered as one of the effective spreading stages in computing area (Murthy, 2009).Virtual reality interaction uses the hand gesture to manipulate the virtual movements using one or two hands for 2D and 3D interactions display (Murthy, 2009).Some virtual reality applications are available in (Bertsch, 2009;Rodriguez, 2008;Guan, 2008).Thomas (2008) worked with multimodal user interfaces that include visual, acoustic and haptic I/O.Based on the Wii controller hardware.

Gesture-to-speech
Gesture-to-speech application which converts hand gestures into speech, this system enables hearing-impaired people to communicate with their surrounding environments through computers and interacts easily with other people even without the knowing for the sign language (Joseph, 1999).Fels (1993) and Fels (1998) introduced Glove Talk system interface between speech synthesizer using data glove device which mapping hand-gestures to speech using neural networks.

Games
For computer games, Xu (2009) applied gesture recognition on virtual game applications.Chambers (2002) used hierarchical recognition of human gestures for sports video annotation.Rautaray (2010) implemented computer vision and gesture recognition techniques, and developed a vision based low cost input device for controlling the VLC player through gestures.

Television Control
Last application for hand postures and gestures is controlling Television devices (Joseph, 1999).Freeman (1995) developed a system to control a television set by hand gestures.Using an open hand and the user can change the channel, turn the television on and off, increase and decrease the volume, and mute the sound.
Table 2. Applications on gesture recognition system Reference Applicaiton (Yin, 2007) a b Human robot interaction.(Yin, 2007) a. humanoid service robot HARO-1; b. virtual robot.(Rautaray, 2010) Gesture interface used to control the VLC player: stop operation stop (Xu, 2009) a b Performing hand gesture to control the virtual game control a. rubik's cub game implementation; b. rubik's cub game.(Freeman, 1995) a b Hand gestures for television control a. user has moved his hand for changing channel; b. channel changed.

Recognition System Methodology
Many researches have been suggested on gesture recognition system for different applications, with different recognition phases but they all agree with the main structure of the gesture recognition system.These phases are segmentation, features detection and extraction, and finally the classification or recognition phase.One of these structures illustrated in Figure 2.

Segmentation
Segmentation phase plays an important role in the system recognition process.Perfect segmentation effects on the accuracy of the recognition system (Hasan, 2010).For any segmentation process, some image processing operations are required for hand shape detection (Murthy, 2009;Hasan, 2010).
Segmentation image algorithms can be classified into two types according to image gray level properties as explained in (Peter, 2011)

Data Acquisition
When the input gesture acquired form colored camera, instrumented glove device or colored glove as shown in Figure 1.The first step is segmentation, to extract the hand region from the input image and isolate it from the background (Hasan, 2010).There are two main methods for object segmentation, first method depends on the color model that can be extracted from the existence RGB color model which could be HSV color model (Hasan, 2010;Hasan, 2011a ;Mo, 2011) or YCbCr color space (Stergiopoulou, 2009); which deals with the pigment of the skin of the human hand (Hasan, 2010), the significant property of this color space is that the human different ethics group can be recognized according to their pigment concentration which can be distinguished according to some skin color saturation (Hasan, 2010).Then, the hand area is isolated from the input gesture with some threshold value.Some normalization for the segmented image might require for obtaining the gestures database which should be invariant against different perturbations like translation, scaling and rotation (Hasan, 2010).The database created with many samples per single gesture, the relation between the number of samples and the accuracy is directly proportional, and between number of samples and the speed is inversely proportional (Hasan, 2010).
Hasan ( 2010) used HSV color model to extract the skin-like hand region by estimating the parameter values for skin pigment, and used Laplacian filter for detection the edges.Stergiopoulou (2009) used YCbCr color model to segment the hand.Maraqa (2008) used color glove for input gestures and HSI color space for the segmentation process.Ghobadi (2008) treated the segmentation process as clustering method by grouping the image pixels among image objects.Lamberti (2011) used HSI color model to segment the hand object.Table 3 shows some applications of the segmentation methods used in the discussed method.

Features Detection and Extraction
The features are the useful information that can be extracted from the segmented hand object by which the machine can understand the meaning of that posture.The numerical representation of these features can be obtained from the vision perspective of the segmented hand object which form the feature extraction phase (Hasan, 2011b).Many researches have been applied to form this feature vector which takes different sizes as well meanings.Hasan (2010) extracted the features vector by dividing the segmented hand object into fixed block size 5x5 brightness value moments; this produce 625 features vector size and only 98 are stored as actual features vector.Stergiopoulou (2009) applied Self-Growing and Self-Organized Neural Gas (SGONG) network to extract the exact shape of the hand region and determine three characteristics as the features; Palm region, Palm center, and Hand slope.Compute angle between the finger root and the hand center named RC Angle, and the joints the fingertip and the hand center named TC, and angle and distance from the palm center.Li (2003) defined a grid of fixed size with 12 blocks gray scale features vector, and each grid cell represents the mean value of the average brightness of the pixels in the block.Lamberti (2011) defined the distance d from the palm to the fingers d i (i = 1, ..., 5), and computed the angle β between the line connecting the centroids of the palm and the fingers, which produces four angles β i (i = 1, ..., 4), so the hand represented by nine numerical features vector (Lamberti, 2011).Table 4 demonstrates features vector representation of these methods.

Recognition
Recognition or classification of hand gestures is the last phase of the recognition system.Hand gestures can be classified using two approaches as mentioned in (Murthy, 2009).
A. Rule based Approaches: which represents the input features as manually encoded rule, and the winner gesture is the one that matched with the encoded rules after his features has been extracted.The main problem of this technique is that the human ability to encode the rules limits the successfulness of the recognition process (Murthy, 2009).B. Machine Learning based Approaches: the most common approaches that considered the gesture as result of some stochastic processes (Murthy, 2009).Most of the problems that based on machine learning have been addressed based on the statistical modeling (Pavlovic, 1997), such as PCA (Kim, 2008), FSM (Verma, 2009).Hidden Markov Models (HMMs) (Keskin, 2003) have been paid attention by many researchers (Murthy, 2009), Kalman filtering (Mo, 2011), Artificial Neural networks (ANNs) (Maraqa, 2008;Murakami, 1999;Fels, 1993;Fels, 1998) which have been utilized in gesture recognition as well.Some researchers used Gaussian distribution for gestures classification (Stergiopoulou, 2009), and Euclidian distance metric (Hasan, 2010).

Conclusion
Human-machine interaction can be achieved by efficient gesture recognition system in which its applications varied from sign language recognition to games and virtual reality interfaces.In this paper a literature review on gesture recognition has been reviewed and analyzed; the major tools for classification process include FSM, PCA, HMMs, and ANNs are discussed.Descriptions of recognition system framework also presented with a demonstration of the main three phases of the recognition system by detection the hand, extraction the features, and recognition the gersture.The major image preprocessing steps necessarily required to features extraction phase are segmentation, edge detection, noise removing, and normalization, these steps may not applied together depending on targeted application.
Figure 1.Ex om image galle

Figure 2 .
Figure2.The flow of gesture recognition system(Moni, 2009) : A. Discontinuity: Which tries to find a mass change in the contrast.B. Similarity: Which computes the similarity between neighbor pixels.
Table 3. Segmentation process from different hand gesture recognition methods image; b. segmented image; c. edge detection.Center of Science and Education 117 angle; b.TC angle; c. distance from the palm center.hand; b. gray scale image partitioned into 12 blocks feature vector.hand object; b. feature vector

Table 1 Althou
csenet.org/cished by Canadian Appearance based approaches: Also known as View Based Approaches, which model the hand using the intensity of 2D images and define the gestures as a sequence of views.Appearance based approaches considered easier than3D model approaches, which led many researchers to search for alternative representations of the hand

Table 4 .
Features representation from different hand gesture recognition methods