Hi everybody, here’s my new Raspberry PI project: the face follower webcam!
When I received my first raspberry, I understood it would have been very funny to play with real world things: so i tried to make the PI react with environment, and I played a lot with speech recognition, various kind of sensors and so on. But then I suddenly realized that the real funny thing would have been to make it see. I also understood that real-time processing a webcam input would be a tough task for a device with those tiny resources.
Now i discovered that this is a tough task, but the raspberry can achieve it, as long as we keep things simple.
As soon as I started playing with webcams, I decided to look for existing libraries to parse the webcam input and to extract some information from it. I soon came across the OpenCV libraries, beautiful libraries to process images from the webcam with a lot of features (which I didn’t fully explore yet), such as face detection.
I started trying to understand if it was possible to make them work on the PI, and yes, someone had already done that, with python too, and it was as easy as a
sudo apt-get install python-opencv
After some tries, i found out that the pi was slow in processing the frames to detect faces, but only because it buffered the frames and I soon had a workaround for that.. and soon I was done: face detection on raspberry PI!
All you need to try face detection on your own is the python-opencv package and the pattern xml file
haarcascade_frontalface_alt.xml, i guess you can find it easily in the Internet.
I started with a script found here and then I modified it for my purposes.
Eventually, I decided to build with my PI a motor driven webcam which could “follow” a face detected by opencv in the webcam stream. I disassembled a lot of things before finding a suitable motor but then I managed to connect it to the Raspberry GPIO (I had to play a little with electronics here, because i didn’t have a servo motor — if you need some information about this part, i’ll be happy to provide it, but i’m not so good with electronics thus the only thing I can say is that it works). Here’s a picture of the circuit:
And here are some photos of the motor, to which I mounted an additional gear wheel.
Once the motor worked, I attached it to a webcam which i plugged into the PI, and then I joined the previous linked script with some GPIO scripting to achieve the goal, here is the result:
import RPi.GPIO as GPIO import time,sys import cv,os from datetime import datetime import Image #GPIO pins i used OUT0 = 11 OUT1 = 13 out0 = False #!enable line: when it's false, the motor turns, when it's true, it stops out1 = False #!the direction the motor turns (clockwise or counter clockwise, it depends on the circuit you made) def DetectFace(image, faceCascade): min_size = (20,20) image_scale = 2 haar_scale = 1.1 min_neighbors = 3 haar_flags = 0 # Allocate the temporary images grayscale = cv.CreateImage((image.width, image.height), 8, 1) smallImage = cv.CreateImage( ( cv.Round(image.width / image_scale), cv.Round(image.height / image_scale) ), 8 ,1) # Convert color input image to grayscale cv.CvtColor(image, grayscale, cv.CV_BGR2GRAY) # Scale input image for faster processing cv.Resize(grayscale, smallImage, cv.CV_INTER_LINEAR) # Equalize the histogram cv.EqualizeHist(smallImage, smallImage) # Detect the faces faces = cv.HaarDetectObjects( smallImage, faceCascade, cv.CreateMemStorage(0), haar_scale, min_neighbors, haar_flags, min_size ) # If faces are found if faces: #os.system("espeak -v it salve") for ((x, y, w, h), n) in faces: return (image_scale*(x+w/2.0)/image.width, image_scale*(y+h/2.0)/image.height) # the input to cv.HaarDetectObjects was resized, so scale the # bounding box of each face and convert it to two CvPoints pt1 = (int(x * image_scale), int(y * image_scale)) pt2 = (int((x + w) * image_scale), int((y + h) * image_scale)) cv.Rectangle(image, pt1, pt2, cv.RGB(255, 0, 0), 5, 8, 0) return False def now(): return str(datetime.now()); def init(): GPIO.setmode(GPIO.BOARD) GPIO.setup(OUT0, GPIO.OUT) GPIO.setup(OUT1, GPIO.OUT) stop() def stop(): #stops the motor and return when it's stopped global out0 if out0 == False: #sleep only if it was moving out0 = True GPIO.output(OUT0,out0) time.sleep(1) else: out0 = True GPIO.output(OUT0,out0) def go(side, t): #turns the motor towards side, for t seconds print "Turning: side: " +str(side) + " time: " +str(t) global out0 global out1 out1 = side GPIO.output(OUT1,out1) out0 = False GPIO.output(OUT0,out0) time.sleep(t) stop() #getting camera number arg cam = 0 if len(sys.argv) == 2: cam = sys.argv #init motor init() capture = cv.CaptureFromCAM(int(cam)) #i had to take the resolution down to 480x320 becuase the pi gave me errors with the default (higher) resolution of the webcam cv.SetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_WIDTH, 480) cv.SetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_HEIGHT, 320) #capture = cv.CaptureFromFile("test.avi") #faceCascade = cv.Load("haarcascades/haarcascade_frontalface_default.xml") #faceCascade = cv.Load("haarcascades/haarcascade_frontalface_alt2.xml") faceCascade = cv.Load("/usr/share/opencv/haarcascades/haarcascade_frontalface_alt.xml") #faceCascade = cv.Load("haarcascades/haarcascade_frontalface_alt_tree.xml") while (cv.WaitKey(500)==-1): print now() + ": Capturing image.." for i in range(5): cv.QueryFrame(capture) #this is the workaround to avoid frame buffering img = cv.QueryFrame(capture) hasFaces = DetectFace(img, faceCascade) if hasFaces == False: print now() + ": Face not detected." else: print now() + ": " + str(hasFaces) val = abs(0.5-hasFaces)/0.5 * 0.3 #print "moving for " + str(val) + " secs" go(hasFaces < 0.5, val) #cv.ShowImage("face detection test", image)
Of course I had to play with timing to make the webcam turn well, and the timings like everything strictly connected to the motor depends on your specific motor and/or circuit.
Here’s the video (in italian):