Gesture and voice control of internet of things : a thesis presented in partial fulfilment of the requirements for the degree of Master of Engineering in Electronics and Computer Engineering at Massey University, Auckland, New Zealand - Han, Xiao (Massey University, 2015)
Nowadays, people's life has been remarkably changed with various intelligent devices which can provide more and more convenient communication with people and with each other. Gesture and voice control are becoming more and more important and widely used. People feel the control system humanized and individualised using biological control. In this thesis, an approach of combined voice and gesture control of Internet of Things is proposed. A prototype is built to show the accuracy and practicality of the system. A Cortex-A8 processor (S5PV210) is used and the embedded Linux version 3.0.8 has been cross-compiled. Qt 4.8.5 has been ported as a UI (User Interface ) framework and OpenCV 2.4.5 employed as vision processing library. Two ZigBee modules are used to provide wireless communication for device control. The system is divided into control station and appliance station. The control station includes development board, USB camera, voice recognition module, LCD screen and ZigBee module. This station is responsible for receiving input signal (from camera or microphone), analyzing the signal and sending control signal to appliance station. The appliance station consists of relay, ZigBee module and appliances. The ZigBee module in the appliance station is to receive control signal and send digital signal to connected relay. The appliance station is a modular unit that can be expanded for multiple appliances. The system can detect and keep tracking user's hand. After recognizing user's gesture, it can control appliances based on certain gestures. Voice control is included as an additional control approach and voice commands can be adjusted for different devices.
Keywords: Automatic speech recognition; Optical data processing; Human-computer interaction; Internet of things