Hand Gesture Recognition Using Average Background and Logical Heuristic Equations

This paper introduces a hand gesture recognition algorithm for Human Computer Interaction using real-time video streaming .The background subtraction technique is used to extract the ROI (Region Of Interest) of the hand. Fingertip is detected using logical heuristics equations that are applied on hand contour, convex hull and convexity defects points. A combination between background subtraction and logical heuristic technique that leads to more accurate results is introduced. Experimental results prove that the proposed algorithm improves the finger's tips detection by 52 % compared to the reference model


INTRODUCTION
Gesture recognition is a challenging area in computer vision. At present, many traditional interfaces such as the keyboard or mouse doesn't provide the best interaction between user and computer. Once hand gestures are recognized , it will make the interaction much better for the user .This paper proposes an algorithm to extract foreground hand regions from background using background averaging and extracting contour, the convex hull and convexity defects information to recognize each gesture. There are different techniques that have been used for hand gesture recognition in different fields. Some of the techniques using skin model , colored gloves , data gloves , PCA , Neural Network. All techniques can be divided into the following categories : Hand segmentation techniques , Feature extraction techniques and Gesture recognition techniques. In this paper a few of the works are discussed.
In papers [1][2][3][4][5] skin color model was used to extract hand pixels. This technique eliminates background pixels or any non-skin colored pixels , then extracting hand region from the image. In others [6][7] external aid like colored glove were used by the researchers for a segmentation purpose .Both of the techniques are very robust, but also vulnerable when objects are present with a color similar to the skin or a glove, since the segmentation in color is based. In papers [8][9][10] data gloves are used , In a paper [8] 14 sensor with a data glove were used to measure inflection points , but accuracy rate was 94%. In the paper [9] data glove package that contains a Bluetooth , 3 accelerometer sensor, and a controller which extract hand gesture features . This paper tests only 3 gestures and the detection speed was very slow. In paper [10] Cyber glove was used to measure angles between fingers , thumb rotation and wrist pitch. This paper test only single gestures. In paper [11] Principal Component Analysis (PCA) was used for feature extraction and pattern recognition with saved images to control a robot in real time. Due to paper results PCA training phase is faster than the neural network based methods which require training database and more computation power. In paper [12] Background averaging and CAMSHIFT algorithms are used for hand tracking in real time and test only two gestures.
Thus a system that could overcome the difficulties faced by various techniques is proposed. The proposed system can detect five gestures. First , foreground objects can be detected using background subtraction. Second, detection of moving objects contours and selection of the biggest contour to track. Third, detection of convex hull and convexity defects points and applies logical heuristics equations to detect finger tips. Finally, after finger tips detection we can calculate how many fingers are stretched, and gestures are used to control an event like mouse actions can be calculated. This paper is organized as follows . Section 2 describes the block diagram of the proposed system. Section 3 describes the steps that we follow to extract hand gestures starting from hand segmentation, feature extraction and gesture recognition. Section 4 describes the proposed system results compared to the reference model. This paper is organized as follows . Section 2 describes the block diagram of the proposed system . Section 3 describes the steps that we follow to extract hand gestures starting from hand segmentation, feature extraction and gesture recognition. Section 4 describes the proposed system results compared to the reference model.

PROPOSED SYSTEM MODEL
The proposed system block diagram is given in Fig.1. this model consists of main four main stages : background subtraction using a background averaging algorithm , contour extraction of all moving objects and select the biggest contour , convexity defects calculations and finger tips detection using logical heuristic equations.

Hand segmentation
As mentioned previously skin color model can be used to detect hand and face regions but it's not efficient in complex backgrounds. In our system we assume that human face won't move so the hands will be the only movable objects in the foreground scene. Thus , background subtraction will be the best solution to find hand area and track hand movements. To get the best background model an average filter is used [12]. Average filter will create an initial background from the first N frame as shown in Eqn (1), as experiment N = 20 The moving areas are obtained by subtracting the current frame from the average background as shown in Eqn(2).

To get better result a threshold filter on the difference image to create a binary image is applied as shown in Eqn (3). Threshold = 50 is used
The updated background is calculated updating the average background's pixels using the equation as shown in Eqn(4) to remove any noise or any static object that was added to the background. O c t 3 0 , 2 0 1 3 Coefficient alpha where is used to control speed of updating process in other words controls how quickly the average background is updated by the current frame.Alpha value is between 0 to 1. 0.001 is used to decrease learning speed. The steps of hand extraction are shown in Fig.3 .

Fig.3 : Steps of background subtraction
The contour is a curve that connects all points that surround specific part of an image that has the same color or intensity.
To identify hand gestures we detect contours of all objects that exist in the threshold image as shown in Fig.4a and select the biggest contour that has the biggest calculated area that represent the hand's area as shown in Fig.4b .
To find the contour Suzuki's algorithm [13] is used .we approximate the filtered contour points to another contour that has fewer number of points. The contour approximation is the process of finding key vertex points [14] to speed up calculations and it consumes low memory size .

Feature extraction
After contour extraction key features to detect hand gestures should to be extracted. In experiment it's found that finger tips are good features to detect hand gestures .
To extract finger tips the extracted contour is analyzed to find convexity information .To understand the shape of an object convex hull and convexity defects information can be used.
To extract convex hulls Sklansky's algorithm in paper [15] is used. it is used to calculate the convex hull for given points. It's ideal for hand gesture analysis and it offers complexity O(n) [16].The algorithm has been applied on the image shown in Fig.4b and the result is shown in Fig.5.
From the convex hull and the original contour convexity defects can be detected which are the holes between contour and convex hull. Fig.6 show start and depth points of each defect where yellow points are start point of defects and red points are the furthest points from contour (depth points). O c t 3 0 , 2 0 1 3

Gesture recognition
After convexity defects points detection all points are filtered and finger tips points are detected using logical heuristic equations [1].This methodology didn't require any training phase . Results are shown in Fig.7

EXPERMINTAL RESULTS
The proposed system is implemented in python 2.7 and tested on AMD Quad-Core processor ( FX 4-Core Processor Black Edition, 3.8 GHz).The test video stream was grabbed from a Web camera with a resolution of 320×240.
There are many techniques for hand gesture recognition. we compared our method with the method proposed in [1] that has used a skin color model and logical heuristics to detect hand gesture . The reference program was tested also under the same environment to get accurate comparison against the proposed algorithm.
The proposed method by background subtraction detection is more robust than the reference technique in [1] in a complex background and simple background with low contrast with skin color as shown in Table[1] ,Table [2] respectively but the two algorithms have the same performance in simple background with high contrast to skin color as shown in Table [3].Each gesture has been tested in different five orientations.   We tested every gestures in 5 different postures and get better performance than the proposed method and improve the performance by 52 %.
The only false positive were observed when the background has many objects with different colors that have intensity value lower than hand pixel's intensities as shown in Fig.8