Controlling servo motors using hand movement (Visual Servoing)

Its been a while since my last entry. My attempts to update this blog more often are to no avail due to the laziness disease which infected me since birth , lack of ideas, and loss of courage, which flowed down the sinkhole. I have no idea what im saying ..really.

Anyway, life are as usual. My latest project i did was to trying to control two servo motors based on the gestures  position of my hand. I did this in order to help two of my student with their FYP. By this, i hope that their project scopes is narrowed, and with that they can start concentrating on other part of the project.

This project consists of using MATLAB as the image processing tools and Arduino UNO to control the movement of the servo.I used Adafruit's motor shield, however it is unnecessary as u can connect two servos directly to the Arduino board. For the webcam, i used Logitech C210 webcam to capture image. Hand position was tracked using simple Background Subtraction method which is quite similar to what i did for my diploma project years ago. In this method, a moving image frame is differentiated with the background image so that only the moving object is produced. The image was then converted from grayscale to a binary image and from there a tracking position of the hand is calculated by simply averaging the white pixels position on both the x-axis and y-axis of the image.This may not track the centroid of the moving object but will give a position near to it.From here the position of the hand is translated into pan-tilt rotation angle using a simple linear conversion and command is sent to the Arduino for servo rotation.

Below is the source code for Arduino:


and here is the source code for MATLAB for acquisition and image processing:



and here is the function required by the above`s source code to calculate the centroid position:



Some lessons learned from this project:

1. USB can't source enough power to run two servos smoothly . I guess my servo used quite a lot of power. Arduino board keeps getting disconnected from the PC's  USB port every time the servos  make a sudden and fast move. I have to move my hands slowly so that Arduino wouldn't get disconnected. Future improvement: use a powered USB hub or external power supply.
2. Using delay in Arduino to overcome this effect will worsen the tracking performance. Data from MATLAB will get accumulated in Arduino's buffer and will be read later after the delay.
3. Different lighting conditions can effect the performance of the tracking system. I have to fiddle round the binary threshold to produce the desired tracking performance (smooth tracking, no noise in the binary image). I used 0.7 in a well lit environment and 0.5 in the opposite.
4. In MATLAB , use fwrite() instead of fprintf() for serial communication. fprintf() only handle ASCII data 0 ~ 127, limiting  the movement of the servo rotation.