i-manager Publications

Indian Sign Language Recognition System using Openpose

Pooja Malhotra*, Chirag K. Maniar**, Nikhil V. Sankpal***, Hardik R. Thakkar****

*-**** Department of Information Technology, K.J. Somaiya College of Engineering, Mumbai University, Mumbai, Maharashtra, India.

Periodicity:June - August'2019
DOI : https://doi.org/10.26634/jcom.7.2.15993

Abstract

Human beings communicate through language, be it verbal or be it a sign language that makes use of body motion. Hearing and Speech impaired people, having no way to communicate verbally, make use of Sign Language. They perform gestures using a sign language in order to convey their message and effectively communicate with each other. Since, not everyone knows about Indian Sign Language (ISL), it becomes difficult for normal people to fluently communicate with Hearing and Speech impaired community. This paper proposes ISL gesture recognition system in order to decrease this communication gap. The dataset consists of videos of ISL gestures, which are performed by different Subjects. The proposed system uses OpenPose library, which helps in creating the skeleton of human body and thus it provides keypoints of the whole human body frame by frame. The use of this library removes the dependency on lighting conditions and background. It helps in focusing on just the gesture movements. After extracting the keypoints, Long Short Term Memory (LSTM) is used for classification of gestures. LSTM model classifies which ISL gesture the particular video belongs to.

Keywords

Indian Sign Language (ISL), OpenPose, Keypoints, Skeleton, Long Short Term Memory (LSTM).

How to Cite this Article?

Malhotra, P., Maniar, C. K., Sankpal, N. V., Thakkar, H. R. (2019). Indian Sign Language Recognition System using Openpose, i-manager's Journal on Computer Science, 7(2), 43-50. https://doi.org/10.26634/jcom.7.2.15993

References

[1]. Caffe. (n.d). Deep learning framework by BAIR. Retrieved from https://caffe.berkeleyvision.org/

[2]. Cao, Z., Hidalgo, G., Simon, T., Wei, S. E., & Sheikh, Y. (2018). OpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields. arXiv preprint arXiv:1812.08008.

[3]. Cao, Z., Simon, T., Wei, S. E., & Sheikh, Y. (2017). Realtime multi-person 2D pose estimation using part affinity fields. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 7291- 7299).

[4]. Gupta, B., Shukla, P., & Mittal, A. (2016, January). K-nearest correlated neighbor classification for Indian Sign Language gesture recognition using feature fusion. In 2016 International Conference on Computer Communication and Informatics (ICCCI) (pp. 1-5). IEEE. doi: 10.1109/IC-CCI.2016.7479951

[5]. Huang, J., Zhou, W., Li, H., & Li, W. (2015, July). Sign language recognition using real-sense. In 2015 IEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP) (pp. 166-170). IEEE. doi: 10.1109/ChinaSIP 2015.7230384

[6]. Kumar, S., & Kaurav, A. (2018, January). Hand gesture through geometric moments (HCI based). In 2018 2^nd International Conference on Inventive Systems and Control (ICISC) (pp. 561-565). IEEE. doi: 10.1109/ICISC.2018.8398862.

[7]. OpenPose (2019). Real-time multi-person keypoint detection library for body, face, hands, and foot estimation. Retrieved from https://github.com/CMUPerceptual- Computing-Lab/openpose

[8]. Simon, T., Joo, H., Matthews, I., & Sheikh, Y. (2017). Hand keypoint detection in single images using multiview bootstrapping. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1145- 1153).

[9]. Singh, A. K., John, B. P., Subramanian, S. V., Kumar, A. S., & Nair, B. B. (2016, December). A low-cost wearable Indian sign language interpretation system. In 2016 International Conference on Robotics and Automation for Humanitarian Applications (RAHA) (pp. 1-6). IEEE. doi: 10.1109/RAHA.2016.7931873.

[10]. Vishwakarma, D. K., & Ansari, S. (2017, November). A framework for human-computer interaction using dynamic time warping and neural network. In 2017 International Conference on Inventive Computing and Informatics (ICICI) (pp. 242-246). IEEE. doi : 10.1109/ICICI.2017.8365346.

[11]. Wei, S. E., Ramakrishna, V., Kanade, T., & Sheikh, Y. (2016). Convolutional pose machines. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4724-4732).

If you have access to this article please login to view the article or kindly login to purchase the article

Single Article

	North Americas,UK, Middle East,Europe		India	Rest of world
	USD	EUR	INR	USD-ROW
Online	15	15

ADD TO CART

Options for accessing this content:

If you would like institutional access to this content, please recommend the title to your librarian.
If you already have i-manager's user account: Login above and proceed to purchase the article.
New Users: Please register, then proceed to purchase the article.