Computer Vision-Based Hand Mouse – Created a hand motion recognition tool that captures hand gestures, extract features, and analyze/recognize the intended user’s activity by using C++ and Open Source Computer Vision library (OpenCV)
The GaussianBlur is performed to remove unwanted noises from the image coming through camera that the smoothly filtered image.
1.2 Color filter (RGB -> YCrCb)
After receiving image with RGB color model, the color filter converts it into YCrCb image.
1.3 Binarization (Binary Image)
Binarization recognizes and distinguishes objects by dividing into black and white using threshold that distinguishes black and white.
1.4 Binary morphology
It is used for pre-processing such as noise elimination or feature extraction using erosion and expansion calculation.
2. Feature extraction
- 2.1 Contour detection (findContours)
- 2.2 Hand contour drawing (drawContours)
2.3 Contour area
It calculates the area inside the contour -> The largest area is recognized as hand -> Save large area only
- 2.4 Drawing square, minimum square, circle based on hand contour
2.5 Central point detection of hand (moments)
It calculates the center of weight central point by calculating the weighted mean
- 2.6 ROI (Region of Interest) processing of hand region
2.7 Central point detection of palm (ROI area)
After drawing a circle with a consistent length based on the center of gravity, count the number of the objects and calculate the number of fingers by holding the furthest coordinate as fingertip point from the center of gravity among the boundary coordinates of the object (finger) outside the circle.
- 2.8 Convex hull, minimum-size polygon including given all dots
- 2.9 Convex hull, Convex defect -> Start point, End point
3. Motion detection
1. Index finger : Mouse Pointer
2. Palm : Drawing / Stop Drawing
3. Fist : Delete Drawing
3.2 Color Selection
Move the fingertip point without moving hands.
Utilize unfolding the palm of hands as a trigger.
1. Unfold the thumb with an angle of ninety degrees. (recognition as a palm)
2. Drawing from the moment that unfolds index finger (or one finger)
Recognize the moment of clenching the fist as a trigger.
Menu should be selected if moving a certain distance from the center coordinate.
1. Clench the fist.
• A cross mark is displayed and divided into quadrants.
• Draw menus in each quadrant (4 menus)
• Basic status / Clear screen / Drawing / Pen thickness
2. Largely move to the direction of the desired menu.
3. Menu selection is completed when moving over a certain distance.
When a menu is selected, a message is shown.
Face recognition and tracking
However, since the CPU can not achieve the desired performance, the Motion History is used to compare the previous image with the current image, and only the area (red square area) that is updated on the screen is calculated. Face detection using the most hardware resources in the program is handled by CUDA operation in the GPU.
- OS : Windows 10 (64bit)
- IDE : Visual Studio 2015
- Language : C++
- Library : OpenCV 3.1.0 / MFC