Hybrid Hand Sign Recognition for Real-Time Wearable Systems with Ambiguity Reduction

Hand sign recognition (HSR) has emerged as a significant field of research and development in the context of wearable systems and human machine interaction. The aim of this research is to investigate the potential of forearm-attached sensors to recognize hand signs and to propose a novel measurement approach for real-time HSR with reduced ambiguities. Three measurement methods are deeply investigated: Force Myography (FMG), Electrical Impedance Tomography (EIT), and surface Electromyography (EMG). The potential of these methods is evaluated in the context of American Sign Language (ASL). For a comprehensive comparative study, it is important to realize same conditions in the data collection. Therefore, a parallel data acquisition interface has been designed for simultaneous data collection. To assess the methods' capacity to distinguish between different hand signs independent of the classification algorithms, we propose a novel method for evaluating the ambiguities between different hand signs directly from the collected data. The application of this method to the collected data for all subjects shows, that EIT and FMG can better differentiate hand signs. Therefore, an FMG-EIT hybrid HSR method is proposed fusing the classification results of both methods based on their complementarity in solving ambiguous cases. The proposed method is able to achieve an average of real time accuracy of 94.16%, 82.5%, and 71.36% for the proposed fusion method, FMG and EIT respectively.