SignBridge: Real-Time Open-Source AI Sign Language Translator for My Deaf Friend

A developer built SignBridge, an open-source bidirectional real-time sign language interpreter, for a deaf friend who communicates primarily through Vietnamese Sign Language. The system runs MediaPipe hand-landmark tracking in the browser via Next.js, streams landmarks over WebSocket to a Spring Boot modular monolith and a FastAPI ONNX inference service that produces glosses, then uses an LLM to assemble natural Vietnamese sentences after a two-second no-sign pause. It targets near-zero-latency translation of continuous gestures into Vietnamese speech and text.

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend https://dev.to/challenges/hacktoberfest-weekend-2026-10-01 I built SignBridge for my close friend Minh, who was born deaf and communicates primarily through Vietnamese Sign Language VSL . In university seminars and daily coffee chats, communication was slow because typing takes away personal eye contact. SignBridge is a bidirectional, real-time sign language interpreter. It runs client-side hand-landmark tracking and local neural network inference to translate continuous hand gestures into natural Vietnamese speech and text with near-zero latency. Hệ thống dịch ngôn ngữ ký hiệu tiếng Việt VSL hai chiều, thời gian thực — đồ án tốt nghiệp 2026. apps/web — Next.js 16: webcam + MediaPipe landmark NGAY TRONG browser → WebSocket apps/core — Spring Boot 4.1 MODULAR MONOLITH Spring Modulith 2.1 : identity | dictionary | translation | dataset | analytics apps/ml — FastAPI: 1 service mỏng suy luận gloss từ chuỗi landmark ONNX training/ — tiền xử lý + notebook huấn luyện Colab/Kaggle, log lên W&B Luồng phiên dịch đầy đủ: webcam → MediaPipe browser → WebSocket → cửa sổ trượt 32 frame → ML service gloss + độ tin cậy → buffer gloss theo phiên → nghỉ tay 2s nhãn NO SIGN → LLM ghép câu tiếng …