{"slug": "building-an-hsk-speaking-test-ai-real-time-tone-grading-with-gemini", "title": "\"Building an HSK Speaking Test AI: Real-time Tone Grading with Gemini", "summary": "A developer built ToneTutor, a free Mandarin speaking assessment tool that uses Google's Gemini AI to grade tone and grammar in real time. The tool records a 3-minute spoken HSK test, estimates the speaker's level on a 1-6 scale, and identifies weak points. The backend processes WebM audio from the browser into WAV format for Gemini, handling iOS Safari limitations.", "body_md": "I built a free Mandarin speaking assessment tool that grades tone + grammar in real time. Here's the engineering behind it.\n\nHSK (Chinese proficiency test) has a speaking component (HSKK), but most learners can't self-assess their level. Online tutors are expensive. Generic AI conversation tools don't grade tones.\n\nSo I built ToneTutor: a 3-minute spoken-HSK test that estimates your speaking level and identifies weak points.\n\n**Frontend:**\n\n**Backend:**\n\n**The Challenge:**\n\nWeb Audio API records as WebM. Gemini expects LINEAR16 (WAV). iOS Safari doesn't support WebM. So:\n\n``` python\npython\nasync def grade_session(transcript: str):\n    prompt = \"\"\"\n    Rate this Mandarin response on HSK 1-6 scale.\n    Assess: tone accuracy, grammar, vocabulary range.\n    Provide: level estimate + weak points.\n    \"\"\"\n    response = await gemini.generate_content(prompt, stream=True)\n    return parse_hsk_level(response)\n\nResults\n\n- 3-min test\n- Real-time feedback\n- Shareable HSK score card\n- Free (limited sessions)\n\nOpen source coming soon. Built because I'm a native speaker + voice actor frustrated with generic tools.\n\nTry it: tonetutor.tefusiang.com (free for 3 sessions)\n\nCurious about the speech-to-text pipeline or tone grading logic? Ask below.\n```\n\n", "url": "https://wpnews.pro/news/building-an-hsk-speaking-test-ai-real-time-tone-grading-with-gemini", "canonical_source": "https://dev.to/64johnlee/building-an-hsk-speaking-test-ai-real-time-tone-grading-with-gemini-5bca", "published_at": "2026-06-28 09:57:22+00:00", "updated_at": "2026-06-28 10:03:47.817327+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "natural-language-processing", "ai-products", "developer-tools"], "entities": ["ToneTutor", "Gemini", "Google", "HSK", "Web Audio API", "iOS Safari"], "alternates": {"html": "https://wpnews.pro/news/building-an-hsk-speaking-test-ai-real-time-tone-grading-with-gemini", "markdown": "https://wpnews.pro/news/building-an-hsk-speaking-test-ai-real-time-tone-grading-with-gemini.md", "text": "https://wpnews.pro/news/building-an-hsk-speaking-test-ai-real-time-tone-grading-with-gemini.txt", "jsonld": "https://wpnews.pro/news/building-an-hsk-speaking-test-ai-real-time-tone-grading-with-gemini.jsonld"}}