生产运营/行业运营>> Hand-Detection-and-Finger-Counting>> 返回
项目作者： kostasthanos

项目描述 ：
A python program for hand detection and finger counting using OpenCV library.
高级语言： Python
项目地址: git://github.com/kostasthanos/Hand-Detection-and-Finger-Counting.git


Hand Detection and Finger Counting

This is a project made with Python and OpenCV library.
For a better and visual understanding of this project and it’s concepts, watch the video in Youtube or click on the image-link below.

Hand-Detection_Finger-Counting

Below there are some instructions and information about the most significant parts of this project.

Preparing the environment

Define a smaller window (test_window) inside the main frame which will be the ROI (Region of Interest).
Only inside this window the tests will be visible.

  1. # Define the Region Of Interest (ROI) window 
  2. top_left = (245, 50) 
  3. bottom_right = (580, 395) 
  4. cv2.rectangle(frame, (top_left[0]-5, top_left[1]-5), (bottom_right[0]+5, bottom_right[1]+5), (0,255,255), 3)



  1. # Frame shape : height and width
  2. h, w = frame.shape[:2] # h, w = 480, 640

Create mask for the hand

Focuse only to user’s hand. So in this part the hand must be isolated from the background.
Apply Gaussian Blur on the ROI.

  1. test_window_blurred = cv2.GaussianBlur(test_window, (5,5), 0)

The window is in BGR format by default. Convert it to HSV format.

  1. # Convert ROI only to HSV format
  2. hsv = cv2.cvtColor(test_window_blurred, cv2.COLOR_BGR2HSV)

In order to find user’s skin color (array values), user can modify the trackbars until the hand is the only thing that is visible. To enable trackbars window someone must define it before starting the program. So after importing the necessary packages add this part of code :

  1. def nothing(x):
  2.     pass
  4. cv2.namedWindow("trackbars")
  5. cv2.createTrackbar("Lower-H", "trackbars", 0, 179, nothing)
  6. cv2.createTrackbar("Lower-S", "trackbars", 0, 255, nothing)
  7. cv2.createTrackbar("Lower-V", "trackbars", 0, 255, nothing)
  8. cv2.createTrackbar("Upper-H", "trackbars", 179, 179, nothing)
  9. cv2.createTrackbar("Upper-S", "trackbars", 255, 255, nothing)
  10. cv2.createTrackbar("Upper-V", "trackbars", 255, 255, nothing)

After that is time to define a range for the colors, based on arrays.

  1. # Find finger (skin) color using trackbars
  2. low_h = cv2.getTrackbarPos("Lower-H", "trackbars")
  3. low_s = cv2.getTrackbarPos("Lower-S", "trackbars")
  4. low_v = cv2.getTrackbarPos("Lower-V", "trackbars")
  5. up_h = cv2.getTrackbarPos("Upper-H", "trackbars")
  6. up_s = cv2.getTrackbarPos("Upper-S", "trackbars")
  7. up_v = cv2.getTrackbarPos("Upper-V", "trackbars")
  9. # Create a range for the colors (skin color) 
  10. lower_color = np.array([low_h, low_s, low_v])
  11. upper_color = np.array([up_h, up_s, up_v])

Finally get the mask.

  1. # Create a mask
  2. mask = cv2.inRange(hsv, lower_color, upper_color)
  3. cv2.imshow("Mask", mask) # Show mask frame



Computing the maximum contour and it’s convex hull

For each frame on the video capture, find the maximum contour inside the ROI.

Max contour

  1. if len(contours) > 0:
  2.     # Find the maximum contour each time (on each frame)
  3.     # --Max Contour--
  4.     max_contour = max(contours, key=cv2.contourArea)
  5.     # Draw maximum contour (blue color)
  6.     cv2.drawContours(test_window, max_contour, -1, (255,0,0), 3)








Convex Hull around max contour

  1. # Find the convex hull "around" the max_contour
  2. # --Convex Hull--
  3. convhull = cv2.convexHull(max_contour, returnPoints = True) 
  4. # Draw convex hull (red color)
  5. cv2.drawContours(test_window, [convhull], -1, (0,0,255), 3, 2)









Finding the point with the minimum y-value

This is the highest point of the convex hull.

  1. min_y = h # Set the minimum y-value equal to frame's height value
  2. final_point = (w, h)
  3. for i in range(len(convhull)):
  4.     point = (convhull[i][0][0], convhull[i][0][1])
  5.     if point[1] < min_y:
  6.     min_y = point[1]
  7.     final_point = point
  8. # Draw a circle (black color) to the point with the minimum y-value
  9. cv2.circle(test_window, final_point, 5, (0,0,0), 2)






Finding the center of the max_contour

The center of max contour is defined by the point (cx, cy) using cv2.moments().

  1.  M = cv2.moments(max_contour) # Moments
  3. # Find the center of the max contour
  4. if M["m00"]!=0:
  5.     cX = int(M["m10"] / M["m00"])
  6.     cY = int(M["m01"] / M["m00"])
  7.     # Draw circle (red color) in the center of max contour
  8.     cv2.circle(test_window, (cX, cY), 6, (0,0,255), 3)






Calculating the Defect points

Contours in OpenCV

Find and draw the polygon that is defined by the contour.

  1. # --Contour Polygon--
  2. contour_poly = cv2.approxPolyDP(max_contour, 0.01*cv2.arcLength(max_contour,True), True)
  3. # Draw contour polygon (white color)
  4. cv2.fillPoly(test_window, [max_contour], text_color)






The result of the command

  1. defects = cv2.convexityDefects(contour_poly, hull)

is an array where each row contains the values:

  1. start point
  2. end point
  3. farthest point
  4. approximate distance to farthest point

Then find these points plus the mid points on each frame as below.

  1. points = []
  2. for i in range(defects.shape[0]): # Len of arrays
  3.     start_index, end_index, far_pt_index, fix_dept = defects[i][0]
  4.     start_pts = tuple(contour_poly[start_index][0])
  5.     end_pts = tuple(contour_poly[end_index][0])
  6.     far_pts = tuple(contour_poly[far_pt_index][0])
  7.     mid_pts = (int((start_pts[0]+end_pts[0])/2), int((start_pts[1]+end_pts[1])/2))   
  8.     points.append(mid_pts)
  10.     #--Start Points-- (yellow color)
  11.     cv2.circle(test_window, start_pts, 2, (0,255,255), 2)








  1. #--End Points-- (black color)
  2. cv2.circle(test_window, end_pts, 2, (0,0,0), 2)








  1. #--Far Points-- (white color)
  2. cv2.circle(test_window, far_pts, 2, text_color, 2)








Finger Counting

In order to do the finger counting we should find a way to check how many fingers are displayed. To do this we are calculating the angle between start point, defect point and end point as shown below.



  1. # --Calculate distances--
  2. # If p1 = (x1, y1) and p2 = (x2, y2) are two points, then the distance between them is
  3.  # Dist : sqrt[(x2-x1)^2 + (y2-y1)^2]
  5. # Distance between the start and the end defect point
  6. a = math.sqrt((end_pts[0] - start_pts[0])**2 + (end_pts[1] - start_pts[1])**2)
  7. # Distance between the farthest (defect) point and the start point
  8. b = math.sqrt((far_pts[0] - start_pts[0])**2 + (far_pts[1] - start_pts[1])**2)
  9. # Distance between the farthest (defect) point and the end point
  10. c = math.sqrt((end_pts[0] - far_pts[0])**2 + (end_pts[1] - far_pts[1])**2)  
  12. angle = math.acos((b**2 + c**2 - a**2) / (2*b*c))  # Find each angle
  13. # If angle > 90 then the farthest point is "outside the area of fingers"
  14. if angle <= 90:  
  15.     count += 1
  16.     frame[0:40, w-40:w] = (0)

Display Finger Counter

  1. for c in range(5):
  2.     if count == c:
  3.         cv2.putText(frame, str(count+1), (w-35,30), font, 2, text_color, 2)
  5. if len(points) <= 1 :
  6.     frame[0:40, w-40:w] = (0)
  7.         cv2.putText(frame, "1", (w-35,30), font, 2, text_color, 2)

Conclusion

This project is aiming on understanding topics such as contours, convex hull, contour polygon. Also it focuses on the defect points which we are finding on the detected hand.

For a better and visual understanding of this project and it’s concepts, watch the video : video.

Author

  • Konstantinos Thanos