Congratulations to George Awad who has successfully defended his thesis and will be awarded the degree of PhD.The title of George's thesis is "A Framework for Sign Language Recognition using Support Vector Machines and Active Learning for Skin Segmentation and Boosted Temporal Sub-units".
He completed his PhD in the School of Computing, DCU under the supervision of Dr. Alistair Sutherland.
Brief Description of Research
This dissertation describes new techniques that can be used in a sign language recognition (SLR) system, and more generally in human gesture systems. Any SLR system consists of three main components: Skin detector, Tracker, and Recognizer. The skin detector is responsible for segmenting skin objects like the face and hands from video frames. The tracker keeps track of the hand location and detects any occlusions that might happen between any skin objects. Finally, the recognizer tries to classify the performed sign into one of the sign classes in our vocabulary using the set of features and information provided by the tracker.
In this work, we propose a new technique for skin segmentation using SVM (support vector machine) active learning combined with region segmentation information. Having segmented the face and hands, we need to track them across the frames. So, we have developed a unified framework for segmenting and tracking skin objects and detecting occlusions. Instead of dealing with the whole sign for recognition, the sign can be broken down into elementary subunits, which are far less in number than the total number of signs in the vocabulary. This motivated us to propose a novel algorithm to model and segment these subunits, then try to learn the informative combinations of subunits/features using a boosting framework. Our results reached above 90% recognition rate using very few training samples.
In summary, we propose a new paradigm to solve the SLR problem by discovering the subunits of the SL then learning the informative ones together with the informative features that can enhance the overall recognition accuracy. We believe that this approach is very promising to scale up the vocabulary of recognizing sign language without compromising the recognition accuracy.
This project was generously funded by the School of Computing, DCU and by Marie Curie Actions.
