Bridging Communication Gaps: The Development of Speech Interpretation and Gesture Notation for the Hearing and Mute Students of South East Asian Institute of Technology, Inc

Communication is fundamental to human interaction, enabling individuals to express thoughts, emotions, and ideas. However, deaf individuals face unique challenges that require alternative methods to bridge the gap between the hearing and non-hearing communities. Despite technological advancements, communication barriers persist within the deaf community. This study aimed to address these challenges by developing the Speech Interpretation and Gesture Notation (S.I.G.N.) application, a communication tool designed to assist both deaf and hearing individuals in understanding each other. The application provides real-time speech and gesture translation, facilitating more accessible interactions in various situations. The development process followed the Incremental Agile Model, allowing each feature of the S.I.G.N. application to be built, tested, and refined iteratively based on ongoing feedback. The application integrates OpenCV and MediaPipe for sign language recognition, ensuring accurate translations. Results indicated that the S.I.G.N. application is both practical and feasible in enhancing interactions between deaf or mute and hearing individuals. It successfully offers a modern and inclusive solution to communication challenges. Therefore, implementing this system is essential to improving accessibility and inclusivity for all users. The System Usability Scale (SUS) evaluation showed an average score of 85.75, indicating that the system was highly usable and effective. Both deaf and mute individuals and hearing users provided positive feedback, confirming that the system functioned as intended. The application successfully facilitated smooth communication by converting gestures to text and speech to gestures. While the system was generally easy to use, some respondents suggested that enhancing real-time translation and expanding the sign language database could further improve the user experience. Overall, the SIGN app met its goal of bridging communication gaps and promoting better interaction between deaf and hearing individuals.