i-manager Publications

Improved Gesture Precision Virtual Personal Assistant (IGP-VPA) System For Speech Impaired People

N. Poornima*, M. Murugan **, Saravana kumar N.***

*,*** Department of Computer Science and Engineering, SRM Valliammai Engineering College, Kancheepuram, Tamil Nadu, India.

** Department of Electronics and Communication Engineering, SRM Valliammai Engineering College, Kancheepuram, Tamil Nadu, India.

Periodicity:June - August'2019
DOI : https://doi.org/10.26634/jpr.6.2.16754

Abstract

The paper proposes an Improved Gesture Precision Virtual Personal Assistant (IGP-VPA) system for speech impaired people that would act as a personal assistant for them. The proposed system first recognizes the sign language performed by the impeded individuals and converts it to text and speech format. The converted speech format will be given as input to the available assistant system such as Amazon Alexa. Then the VPA system responds to the speech, which would act as a personal assistant for speech impaired people. The previous method, Static and Dynamic Hand Gesture Recognition in Depth Data using Dynamic Time Warping also helps the speech impaired persons with 71.9% of gesture identification precision. To increase the gesture identification precision, IGP-VPA has been proposed. IGP-VPA uses iterative optimization process using Convolutional Neural Network (CNN), which overcomes drawbacks, such as memory intensive storage problem, poor generalization, and performance drops when training with voluminous samples. IGP-VPA is trained with 2262 gestures, each with approximately 1500 image. Indoctrinating with the huge dataset makes a difference to realize the precision of the recognition. The experimental results concluded the average precision in gesture identification as 82.9%, which is 11.78% higher compared to the Static and Dynamic Hand Gesture Recognition in Depth Data using Dynamic Time Warping.

Keywords

Gesture, Convolutional Neural Network (CNN), Single Shot Multi Box Detector (SSD).

How to Cite this Article?

Poornima, N., Murugan, M., & Kumar, N. S. (2019). Improved Gesture Precision Virtual Personal Assistant (IGP-VPA) System for Speech Impaired People. i-manager’s Journal on Pattern Recognition, 6(2), 17-24. https://doi.org/10.26634/jpr.6.2.16754

References

[1]. Aiswarya, V., Raju, N. N., Joy, S. S. J., Nagarajan, T., & Vijayalakshmi, P. (2018, March). Hidden Markov Model-based Sign Language to Speech Conversion System In Tamil. In 2018 Fourth International Conference on Biosignals, Images and Instrumentation (ICBSII) (pp. 206- 212). IEEE. https://doi.org/10.1109/ICBSII.2018.8524802

[2]. Avola, D., Bernardi, M., Cinque, L., Foresti, G. L., & Massaroni, C. (2018). Exploiting recurrent neural networks and leap motion controller for the recognition of sign language and semaphoric hand gestures. IEEE Transactions on Multimedia, 21(1), 234-245. https://doi. org/10.1109/TMM.2018.2856094

[3]. Bao, P., Maqueda, A. I., del-Blanco, C. R., & García, N. (2017). Tiny hand gesture recognition without localization via a deep convolutional network. IEEE Transactions on Consumer Electronics, 63(3), 251-257. https://doi.org/ 10.1109/tce.2017.014971

[4]. Chakraborty, B. K., Sarma, D., Bhuyan, M. K., & MacDorman, K. F. (2017). Review of constraints on vision-based gesture recognition for Human-Computer Interaction. IET Computer Vision, 12(1), 3-15. https://doi. org/10.1049/iet-cvi.2017.0052

[5]. Cui, R., Liu, H., & Zhang, C. (2019). A deep neural framework for continuous sign language recognition by iterative training. IEEE Transactions on Multimedia, 21( 7), 1880-1891. https://doi.org/10.1109/TMM.2018.2889563

[6]. Dreuw, P., Rybach, D., Deselaers, T., Zahedi, M., & Ney, H. (2007). Speech recognition techniques for a sign language recognition system. In Eighth Annual Conference of the International Speech Communication Association, (pp. 2513-2516).

[7]. Elpeltagy, M., Abdelwahab, M., Hussein, M. E., Shoukry, A., Shoala, A., & Galal, M. (2018). Multi-modality- based Arabic sign language recognition. IET Computer Vision, 12(7), 1031-1039. https://doi.org/10. 1049/iet-cvi.2017.0598

[8]. Ghotkar, A. S., Khatal, R., Khupase, S., Asati, S., & Hadap, M. (2012, January). Hand gesture recognition for Indian Sign Language. In 2012 International Conference on Computer Communication and Informatics (pp. 1-4). IEEE. https://doi.org/10.1109/ICCCI.2012.6158807

[9]. Hussain, S., Saxena, R., Han, X., Khan, J. A., & Shin, H. (2017). Hand gesture recognition using deep learning. 2017 International SoC Design Conference (ISOCC). https://doi.org/10.1109/isocc.2017.8368821

[10]. Jimenez, J., Martin, A., Uc, V., & Espinosa, A. (2017). Mexican Sign Language Alphanumerical gestures recognition using 3D Haar-like Features. IEEE Latin America Transactions, 15(10), 2000-2005. https://doi.org/ 10.1109/TLA.2017.8071247

[11]. Lee, D., Yoon, H., & Kim, J. (2016). Continuous gesture recognition by using gesture spotting. 2016 16^th International Conference on Control, Automation and Systems (ICCAS). https://doi.org/10.1109/iccas.2016.78 32502

[12]. Shenoy, K., Dastane, T., Rao, V., & Vyavaharkar, D. (2018, July). Real-time Indian Sign Language (ISL) Recognition. In 2018 9^th International Conference on Computing, Communication and Networking Technologies (ICCCNT) (pp. 1-9). IEEE. https://doi.org/ 10.1109/ICCCNT.2018.8493808

[13]. Simão, M. A., Neto, P., & Gibaru, O. (2016). Unsupervised gesture segmentation by motion detection of a real-time data stream. IEEE Transactions on Industrial Informatics, 13(2), 473-481. https://doi.org/10.1109/TII. 2016.2613683

[14]. Truong, V. N., Yang, C. K., & Tran, Q. V. (2016, October). A translator for American sign language to text and speech. In 2016 IEEE 5^th Global Conference on Consumer Electronics (pp. 1-2). IEEE. https://doi.org/10. 1109/GCCE.2016.7800427

[15]. Wu, D., Pigou, L., Kindermans, P.-J., Le, N. D.-H., Shao, L., Dambre, J., & Odobez, J.-M. (2016). Deep dynamic neural networks for multimodal gesture segmentation and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(8), 1583- 1597. https://doi.org/ 10.1109/tpami.2016.2537340

[16]. Wu, Y., Wu, Z., & Fu, C. (2018). Continuous Arm Gesture Recognition Based on Natural Features and Logistic Regression. IEEE Sensors Journal, 18(19), 8143- 8153. https://doi.org/10.1109/JSEN.2018.2863044

If you have access to this article please login to view the article or kindly login to purchase the article

Single Article

	North Americas,UK, Middle East,Europe		India	Rest of world
	USD	EUR	INR	USD-ROW
Online	15	15

ADD TO CART

Options for accessing this content:

If you would like institutional access to this content, please recommend the title to your librarian.
If you already have i-manager's user account: Login above and proceed to purchase the article.
New Users: Please register, then proceed to purchase the article.