Voice-First UX: Building the React Frontend and Web Speech Integration for BizBoost AI At the AI for Bharat Hackathon, a developer built BizBoost AI, a voice-first React frontend for Indian merchants that uses the browser's native Web Speech API for Hindi speech recognition. The frontend integrates with a serverless AWS backend via Amazon API Gateway, Lambda, DynamoDB, and Amazon Bedrock to generate product descriptions from spoken input. The project aims to make AI accessible to local shopkeepers who cannot type or navigate complex interfaces. When we talk about the power of Generative AI, we often focus entirely on the backend models. We talk about parameters, tokens, context windows, and hosting servers. But for millions of local shopkeepers and micro-merchants across India, these technical terms don't mean anything. For them, technology is only useful if it solves their day-to-day problems with minimal effort. During the AI for Bharat Hackathon organized by Hack2Skill and supported by AWS , my teammate Mohammed Ayaan Adil Ahmed and I wanted to bridge this UX gap. We built BizBoost AI — बोल के बेचो Speak it. Sell it. 🎙️ While Ayaan engineered the serverless AWS backend and Amazon Bedrock integration, I was responsible for designing and building the frontend. My goal was simple: create an ultra-simple, mobile-first, voice-driven interface that anyone could use without training. Here is how I implemented browser-based Hindi speech recognition and optimized the user experience for Indian merchants. Local business owners are busy. If you force them to type out long product descriptions, navigate complex navigation tabs, or manually translate copy from Hindi to English, they will quickly log off. The design principles I established for the frontend were: Rather than overloading our React application with heavy, paid third-party voice APIs, I integrated the browser's native Web Speech API . It supports Hindi hi-IN transcription right out of the box with incredible accuracy, and costs absolutely nothing. Here is the React hook I wrote to manage the recording state, language settings, and transcription capturing: js import { useState, useEffect } from 'react'; export const useSpeechToText = = { const transcript, setTranscript = useState "" ; const isListening, setIsListening = useState false ; const error, setError = useState null ; const startListening = = { setError null ; const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition; if SpeechRecognition { setError "Speech recognition is not supported on this browser. Try Google Chrome." ; return; } const recognition = new SpeechRecognition ; recognition.lang = 'hi-IN'; // Sets spoken language to Hindi / Hinglish recognition.continuous = false; // Stop recording once the user pauses speaking recognition.interimResults = false; // Only deliver final results recognition.onstart = = { setIsListening true ; }; recognition.onend = = { setIsListening false ; }; recognition.onerror = event = { console.error "Speech Recognition Error:", event.error ; setError Error: ${event.error}. Please try again. ; setIsListening false ; }; recognition.onresult = event = { const currentTranscript = event.results 0 0 .transcript; setTranscript currentTranscript ; }; recognition.start ; }; return { transcript, isListening, startListening, error }; }; Once we capture the spoken transcript locally in React, we send a secure HTTP POST request to the serverless backend designed by Ayaan. Here is how the frontend state orchestrates with the broader serverless system topology: | Architecture Layer | Service / Component | Description | |---|---|---| 📱 Frontend Interface | React App Hosted on AWS Amplify | Captures user audio inputs via native Web Speech API and establishes mobile-responsive layouts. | 🌐 API Gateway Integration | Amazon API Gateway | Manages rate limiting, handles CORS configurations, and maps secure HTTPS endpoints. | ⚡ Serverless Compute Layer | AWS Lambda Python | Parses string inputs, builds dynamic system prompts, handles errors, and calls Bedrock runtimes. | 🗄️ Database Management | Amazon DynamoDB | Acts as an ultra-fast historical log registry to store and fetch user session histories. | 🧠 Generative AI Core Engine | Amazon Bedrock Amazon Nova Lite | Understands multi-dialect Hinglish phrases and structures raw logic payloads into clean, dual-language outputs. | I set up the state machine in React to update the UI dynamically, ensuring that the merchant sees a friendly loading state instead of an awkward blank space while Amazon Bedrock Nova generates the marketing post. python import React, { useState, useEffect } from 'react'; import { useSpeechToText } from './hooks/useSpeechToText'; function App { const { transcript, isListening, startListening, error } = useSpeechToText ; const loading, setLoading = useState false ; const posts, setPosts = useState { hindi post: "", english post: "" } ; const handleGenerateCopy = async textToProcess = { setLoading true ; try { const response = await fetch 'https://YOUR API GATEWAY URL/generate', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify { transcript: textToProcess } } ; const data = await response.json ; setPosts { hindi post: data.hindi post, english post: data.english post } ; } catch err { console.error "API Error:", err ; } finally { setLoading false ; } }; // Automatically trigger API call once voice transcription is ready useEffect = { if transcript { handleGenerateCopy transcript ; } }, transcript ; return