Contactless Mouse and Voice System using ML and AI Integration

Kaki Leela Prasad*, Palle Suresh Naidu**, Ravada Harsha Vardhan ***, Reddy Sresta Sree ****, Yeluri Prem Bharath *****
*-***** Department of Computer Science and Engineering, Maharaj Vijayaram Gajapathi Raj College of Engineering, Vizianagaram, Andhra Pradesh, India.
Periodicity:January - March'2024
DOI : https://doi.org/10.26634/jcom.11.4.20559

Abstract

The aim of this paper is to greatly decrease the use of physical devices. To achieve this, the approach of human- computer interaction was employed, implementing a system that uses only finger movements to control the mouse pointer on the screen without relying on any hardware like a physical mouse. This system uses OpenCV as a medium for input obtained from a live camera and uses media pipe models to convert plane inputs from OpenCV into useful data, which is more efficient compared to traditional models like object and color detection. Along with the contactless mouse, this paper also implemented a voice system to support the contactless mouse in its activation and deactivation. This is achieved with the help of modules like speech recognition and PyAudio. This system also assists the user with simple tasks, such as opening Google or browsing content. By combining both voice recognition and a contactless mouse using hand movements, the system was converted into a seemingly contactless device.

Keywords

Hand Landmarking Model (HLM), Human-Computer Interaction (HCI), Media Pipe, OpenCV, PyAutoGUI, PyAudio, Speech Recognition, Text-to-Speech (TTS) Engine.

How to Cite this Article?

Prasad, K. l., Naidu, P. S., Vardhan, R. H., Sree, R. S., and Bharath, Y. P. (2024). Contactless Mouse and Voice System using ML and AI Integration. i-manager’s Journal on Computer Science, 11(4), 1-11. https://doi.org/10.26634/jcom.11.4.20559

References

[2]. Abougarair, A. J., Aburakhis, M. K., & Zaroug, M. (2022). Design and implementation of smart voice assistant and recognizing academic words. International Robotics & Automation Journal, 8(1), 27-32.
[4]. Gaikwad, S. K., Gawali, B. W., & Yannawar, P. (2010). A review on speech recognition technique. International Journal of Computer Applications, 10(3), 16-24.
[5]. Jyothi, C. R., Beracah, P., & Gowtham, M. (2021). Voice assistant in accessing real world applications. Journal of Engineering Science (JES), 12(7), 333-335.
[8]. Pandey, D., Ali, A., Dubey, S., Srivastava, M., Dwivedi, S., & Raza, M. S. (2022). Voice assistant using Python and AI. International Research Journal of Engineering and Technology, 9(05), 832-838.
[9]. Pundir, N. (2019). Voice Recognition (AI): Voice assistant robot. International Research Journal of Engineering and Technology (Irjet), 6 (6), 3824-3827.
[18]. Zhang, Z., Wu, Y., Shan, Y., & Shafer, S. (2001, November). Visual panel: Virtual mouse, keyboard and 3D controller with an ordinary piece of paper. In Proceedings of the 2001 Workshop on Perceptive User Interfaces (pp. 1-8).
If you have access to this article please login to view the article or kindly login to purchase the article

Purchase Instant Access

Single Article

North Americas,UK,
Middle East,Europe
India Rest of world
USD EUR INR USD-ROW
Online 15 15

Options for accessing this content:
  • If you would like institutional access to this content, please recommend the title to your librarian.
    Library Recommendation Form
  • If you already have i-manager's user account: Login above and proceed to purchase the article.
  • New Users: Please register, then proceed to purchase the article.