Dynamic Training of Hand Gesture Recognition System Attila Licsár, Tamás Szirányi University of Veszprém, Hungary Department of Image Processing and Neurocomputing

Motivations  Effective human-computer interface Camera-projector system

 User-interface controlled by hand gestures Hand gesture recognition

 User-independence: any user should work with the system with high recognition rate. Interactive gesture training

Camera-Projector System Front-projected system

Desktop image handled by the operating system

Information area

Monitor

User

Recognition area

interface

Settings panel of the application

Projector image

Computer equipped with dual-head video card

Projector

Camera

Camera image

System Overview Image grabbing

Displaying refreshed image by projector

background Background & image updating

Projector image

forearm segmentation

arm mask Gesture analysing

Displayed image modification by Finite State Machine

Go to the next slide

Geometrical Distorsion of the Camera Image Possible geometric distorsions: translation, uniform scaling, shearing, keystoning, non-uniform scaling, bending.

Point registration

Camera (distorted) image (x,y)

Projector (input) image (x’,y’)

Coordinate registration between camera and projector images Image warping by second order polynomial equations x  a0  a1  x  a2  y  a3  x 2  a4  xy  a5  y 2 y  b0  b1  x  b2  y  b3  x 2  b4  xy  b5  y 2

Segmentation Processes Problem: hand surface reflects the projected background Background subtraction reference background image human skin partly absorbs the light Forearm segmentation: boundary-based classification

Camera image

Segmented arm mask

Segmented contour

Segmentation with Changing Background Changed image to be projected

Camera image

Warped projector image

Background differencing by artificially generated background

Color corrected projector image

Segmented image

Automatic Wrist Point Detection Based on Wrist Width Wrist detection condition:

w5

w1

wi