Voice-First UX: Building the React Frontend and Web Speech Integration for BizBoost AI

When we talk about the power of Generative AI, we often focus entirely on the backend models. We talk about parameters, tokens, context windows, and hosting servers. But for millions of local shopkeepers and micro-merchants across India, these technical terms don't mean anything. For them, technology is only useful if it solves their day-to-day problems with minimal effort.

During the AI for Bharat Hackathon (organized by Hack2Skill and supported by AWS), my teammate Mohammed Ayaan Adil Ahmed and I wanted to bridge this UX gap.

We built BizBoost AI — बोल के बेचो (Speak it. Sell it.) 🎙️

While Ayaan engineered the serverless AWS backend and Amazon Bedrock integration, I was responsible for designing and building the frontend. My goal was simple: create an ultra-simple, mobile-first, voice-driven interface that anyone could use without training.

Here is how I implemented browser-based Hindi speech recognition and optimized the user experience for Indian merchants.

Local business owners are busy. If you force them to type out long product descriptions, navigate complex navigation tabs, or manually translate copy from Hindi to English, they will quickly log off.

The design principles I established for the frontend were:

Rather than over our React application with heavy, paid third-party voice APIs, I integrated the browser's native Web Speech API. It supports Hindi (hi-IN

) transcription right out of the box with incredible accuracy, and costs absolutely nothing.

Here is the React hook I wrote to manage the recording state, language settings, and transcription capturing:

import { useState, useEffect } from 'react';

export const useSpeechToText = () => {
  const [transcript, setTranscript] = useState("");
  const [isListening, setIsListening] = useState(false);
  const [error, setError] = useState(null);

  const startListening = () => {
    setError(null);
    const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition;

    if (!SpeechRecognition) {
      setError("Speech recognition is not supported on this browser. Try Google Chrome.");
      return;
    }

    const recognition = new SpeechRecognition();
    recognition.lang = 'hi-IN'; // Sets spoken language to Hindi / Hinglish
    recognition.continuous = false; // Stop recording once the user s speaking
    recognition.interimResults = false; // Only deliver final results

    recognition.onstart = () => {
      setIsListening(true);
    };

    recognition.onend = () => {
      setIsListening(false);
    };

    recognition.onerror = (event) => {
      console.error("Speech Recognition Error:", event.error);
      setError(`Error: ${event.error}. Please try again.`);
      setIsListening(false);
    };

    recognition.onresult = (event) => {
      const currentTranscript = event.results[0][0].transcript;
      setTranscript(currentTranscript);
    };

    recognition.start();
  };

  return { transcript, isListening, startListening, error };
};

Once we capture the spoken transcript locally in React, we send a secure HTTP POST request to the serverless backend designed by Ayaan. Here is how the frontend state orchestrates with the broader serverless system topology:

I set up the state machine in React to update the UI dynamically, ensuring that the merchant sees a friendly state (instead of an awkward blank space) while Amazon Bedrock Nova generates the marketing post.

import React, { useState, useEffect } from 'react';
import { useSpeechToText } from './hooks/useSpeechToText';

function App() {
  const { transcript, isListening, startListening, error } = useSpeechToText();
  const [, set] = useState(false);
  const [posts, setPosts] = useState({ hindi_post: "", english_post: "" });

  const handleGenerateCopy = async (textToProcess) => {
    set(true);
    try {
      const response = await fetch('https://YOUR_API_GATEWAY_URL/generate', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ transcript: textToProcess })
      });

      const data = await response.json();
      setPosts({
        hindi_post: data.hindi_post,
        english_post: data.english_post
      });
    } catch (err) {
      console.error("API Error:", err);
    } finally {
      set(false);
    }
  };

  // Automatically trigger API call once voice transcription is ready
  useEffect(() => {
    if (transcript) {
      handleGenerateCopy(transcript);
    }
  }, [transcript]);

  return (
    <div className="app-container">
      {/* UI Elements & Buttons */}
    </div>
  );
}

export default App;

Because our target users operate in physical market stalls, we optimized the app for mobile devices.

The AI for Bharat Hackathon showed me that UI/UX is the ultimate bridge between powerful artificial intelligence and real-world utility. Working with Ayaan was incredible—as he optimized our serverless backend to keep latencies low, I made sure the user interface made those quick response times feel natural.

By placing voice and native language support at the center of the design, we built a tool that respects the workflows of India's micro-merchants rather than trying to change them.

What design patterns do you use when building accessible AI applications? Let's swap ideas in the comments below! 🚀