Sign to Speech: A Machine Learning Approach for Deaf and Mute Communication

This research demonstrates a novel attempt to help people who are both deaf and mute by creating a communication assist system that translates hand signs into words. The system uses a camera to capture hand movements and the trained recognition model identifies them. After recognition, text translation followed by speech synthesis through a voice module is performed. To train and evaluate the system, a custom dataset capturing common gestures was created. The sign-to-speech solution is tailored to operate on constrained, cost-effective hardware such as smartphones and tablets. Furthermore, this review discusses the commonly used datasets in sign-to-speech research and their limitations in terms of size, diversity, and standardization. It also suggests a general flow of implementation starting from data collection, preprocessing, feature extraction, model training, and conversion to speech. The paper highlights key challenges such as gesture variability, occlusion, and real-time processing.

@article{ijter2025212508212521,
  author = {Dr. Hetal Bhaidasna and Dr. Zubin Bhaidasna},
  title = {Sign to Speech: A Machine Learning Approach for Deaf and Mute Communication},
  journal = {International Journal of Technology &  Emerging Research },
  year = {2025},
  volume = {1},
  number = {4},
  pages = {80-84},
  issn = {3068-109X},
  url = {https://www.ijter.org/article/212508212521/sign-to-speech-a-machine-learning-approach-for-deaf-and-mute-communication},
  abstract = {This research demonstrates a novel attempt to help people who are both deaf and mute by creating a communication assist system that translates hand signs into words. The system uses a camera to capture hand movements and the trained recognition model identifies them. After recognition, text translation followed by speech synthesis through a voice module is performed. To train and evaluate the system, a custom dataset capturing common gestures was created. The sign-to-speech solution is tailored to operate on constrained, cost-effective hardware such as smartphones and tablets. Furthermore, this review discusses the commonly used datasets in sign-to-speech research and their limitations in terms of size, diversity, and standardization. It also suggests a general flow of implementation starting from data collection, preprocessing, feature extraction, model training, and conversion to speech. The paper highlights key challenges such as gesture variability, occlusion, and real-time processing.},
  keywords = {Sign Language, Machine Learning, CNN},
  month = {Aug},
}

Copyright © 2025 Authors retain the copyright of this article. This article is an open access article distributed under the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.