{"slug": "voice-first-ux-building-the-react-frontend-and-web-speech-integration-for-ai", "title": "Voice-First UX: Building the React Frontend and Web Speech Integration for BizBoost AI", "summary": "At the AI for Bharat Hackathon, a developer built BizBoost AI, a voice-first React frontend for Indian merchants that uses the browser's native Web Speech API for Hindi speech recognition. The frontend integrates with a serverless AWS backend via Amazon API Gateway, Lambda, DynamoDB, and Amazon Bedrock to generate product descriptions from spoken input. The project aims to make AI accessible to local shopkeepers who cannot type or navigate complex interfaces.", "body_md": "When we talk about the power of Generative AI, we often focus entirely on the backend models. We talk about parameters, tokens, context windows, and hosting servers. But for millions of local shopkeepers and micro-merchants across India, these technical terms don't mean anything. For them, technology is only useful if it solves their day-to-day problems with minimal effort.\n\nDuring the **AI for Bharat Hackathon** (organized by Hack2Skill and supported by AWS), my teammate **Mohammed Ayaan Adil Ahmed** and I wanted to bridge this UX gap.\n\nWe built **BizBoost AI — बोल के बेचो (Speak it. Sell it.)** 🎙️\n\nWhile Ayaan engineered the serverless AWS backend and Amazon Bedrock integration, I was responsible for designing and building the frontend. My goal was simple: create an ultra-simple, mobile-first, voice-driven interface that anyone could use without training.\n\nHere is how I implemented browser-based Hindi speech recognition and optimized the user experience for Indian merchants.\n\nLocal business owners are busy. If you force them to type out long product descriptions, navigate complex navigation tabs, or manually translate copy from Hindi to English, they will quickly log off.\n\nThe design principles I established for the frontend were:\n\nRather than overloading our React application with heavy, paid third-party voice APIs, I integrated the browser's native **Web Speech API**. It supports Hindi (`hi-IN`\n\n) transcription right out of the box with incredible accuracy, and costs absolutely nothing.\n\nHere is the React hook I wrote to manage the recording state, language settings, and transcription capturing:\n\n``` js\nimport { useState, useEffect } from 'react';\n\nexport const useSpeechToText = () => {\n  const [transcript, setTranscript] = useState(\"\");\n  const [isListening, setIsListening] = useState(false);\n  const [error, setError] = useState(null);\n\n  const startListening = () => {\n    setError(null);\n    const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition;\n\n    if (!SpeechRecognition) {\n      setError(\"Speech recognition is not supported on this browser. Try Google Chrome.\");\n      return;\n    }\n\n    const recognition = new SpeechRecognition();\n    recognition.lang = 'hi-IN'; // Sets spoken language to Hindi / Hinglish\n    recognition.continuous = false; // Stop recording once the user pauses speaking\n    recognition.interimResults = false; // Only deliver final results\n\n    recognition.onstart = () => {\n      setIsListening(true);\n    };\n\n    recognition.onend = () => {\n      setIsListening(false);\n    };\n\n    recognition.onerror = (event) => {\n      console.error(\"Speech Recognition Error:\", event.error);\n      setError(`Error: ${event.error}. Please try again.`);\n      setIsListening(false);\n    };\n\n    recognition.onresult = (event) => {\n      const currentTranscript = event.results[0][0].transcript;\n      setTranscript(currentTranscript);\n    };\n\n    recognition.start();\n  };\n\n  return { transcript, isListening, startListening, error };\n};\n```\n\nOnce we capture the spoken transcript locally in React, we send a secure HTTP POST request to the serverless backend designed by Ayaan. Here is how the frontend state orchestrates with the broader serverless system topology:\n\n| Architecture Layer | Service / Component | Description |\n|---|---|---|\n📱 Frontend Interface |\nReact App Hosted on AWS Amplify\n|\nCaptures user audio inputs via native Web Speech API and establishes mobile-responsive layouts. |\n🌐 API Gateway Integration |\nAmazon API Gateway |\nManages rate limiting, handles CORS configurations, and maps secure HTTPS endpoints. |\n⚡ Serverless Compute Layer |\nAWS Lambda (Python) |\nParses string inputs, builds dynamic system prompts, handles errors, and calls Bedrock runtimes. |\n🗄️ Database Management |\nAmazon DynamoDB |\nActs as an ultra-fast historical log registry to store and fetch user session histories. |\n🧠 Generative AI Core Engine |\nAmazon Bedrock (Amazon Nova Lite) |\nUnderstands multi-dialect Hinglish phrases and structures raw logic payloads into clean, dual-language outputs. |\n\nI set up the state machine in React to update the UI dynamically, ensuring that the merchant sees a friendly loading state (instead of an awkward blank space) while Amazon Bedrock Nova generates the marketing post.\n\n``` python\nimport React, { useState, useEffect } from 'react';\nimport { useSpeechToText } from './hooks/useSpeechToText';\n\nfunction App() {\n  const { transcript, isListening, startListening, error } = useSpeechToText();\n  const [loading, setLoading] = useState(false);\n  const [posts, setPosts] = useState({ hindi_post: \"\", english_post: \"\" });\n\n  const handleGenerateCopy = async (textToProcess) => {\n    setLoading(true);\n    try {\n      const response = await fetch('https://YOUR_API_GATEWAY_URL/generate', {\n        method: 'POST',\n        headers: { 'Content-Type': 'application/json' },\n        body: JSON.stringify({ transcript: textToProcess })\n      });\n\n      const data = await response.json();\n      setPosts({\n        hindi_post: data.hindi_post,\n        english_post: data.english_post\n      });\n    } catch (err) {\n      console.error(\"API Error:\", err);\n    } finally {\n      setLoading(false);\n    }\n  };\n\n  // Automatically trigger API call once voice transcription is ready\n  useEffect(() => {\n    if (transcript) {\n      handleGenerateCopy(transcript);\n    }\n  }, [transcript]);\n\n  return (\n    <div className=\"app-container\">\n      {/* UI Elements & Buttons */}\n    </div>\n  );\n}\n\nexport default App;\n```\n\nBecause our target users operate in physical market stalls, we optimized the app for mobile devices.\n\nThe AI for Bharat Hackathon showed me that UI/UX is the ultimate bridge between powerful artificial intelligence and real-world utility. Working with Ayaan was incredible—as he optimized our serverless backend to keep latencies low, I made sure the user interface made those quick response times feel natural.\n\nBy placing voice and native language support at the center of the design, we built a tool that respects the workflows of India's micro-merchants rather than trying to change them.\n\nWhat design patterns do you use when building accessible AI applications? Let's swap ideas in the comments below! 🚀", "url": "https://wpnews.pro/news/voice-first-ux-building-the-react-frontend-and-web-speech-integration-for-ai", "canonical_source": "https://dev.to/sufiya_shariff/voice-first-ux-building-the-react-frontend-and-web-speech-integration-for-bizboost-ai-5apg", "published_at": "2026-07-17 07:31:56+00:00", "updated_at": "2026-07-17 08:01:23.231723+00:00", "lang": "en", "topics": ["artificial-intelligence", "generative-ai", "developer-tools", "natural-language-processing", "ai-products"], "entities": ["BizBoost AI", "Mohammed Ayaan Adil Ahmed", "AWS", "Amazon Bedrock", "Amazon API Gateway", "AWS Lambda", "Amazon DynamoDB", "Amazon Nova Lite"], "alternates": {"html": "https://wpnews.pro/news/voice-first-ux-building-the-react-frontend-and-web-speech-integration-for-ai", "markdown": "https://wpnews.pro/news/voice-first-ux-building-the-react-frontend-and-web-speech-integration-for-ai.md", "text": "https://wpnews.pro/news/voice-first-ux-building-the-react-frontend-and-web-speech-integration-for-ai.txt", "jsonld": "https://wpnews.pro/news/voice-first-ux-building-the-react-frontend-and-web-speech-integration-for-ai.jsonld"}}