"Building an HSK Speaking Test AI: Real-time Tone Grading with Gemini

A developer built ToneTutor, a free Mandarin speaking assessment tool that uses Google's Gemini AI to grade tone and grammar in real time. The tool records a 3-minute spoken HSK test, estimates the speaker's level on a 1-6 scale, and identifies weak points. The backend processes WebM audio from the browser into WAV format for Gemini, handling iOS Safari limitations.

I built a free Mandarin speaking assessment tool that grades tone + grammar in real time. Here's the engineering behind it. HSK Chinese proficiency test has a speaking component HSKK , but most learners can't self-assess their level. Online tutors are expensive. Generic AI conversation tools don't grade tones. So I built ToneTutor: a 3-minute spoken-HSK test that estimates your speaking level and identifies weak points. Frontend: Backend: The Challenge: Web Audio API records as WebM. Gemini expects LINEAR16 WAV . iOS Safari doesn't support WebM. So: python python async def grade session transcript: str : prompt = """ Rate this Mandarin response on HSK 1-6 scale. Assess: tone accuracy, grammar, vocabulary range. Provide: level estimate + weak points. """ response = await gemini.generate content prompt, stream=True return parse hsk level response Results - 3-min test - Real-time feedback - Shareable HSK score card - Free limited sessions Open source coming soon. Built because I'm a native speaker + voice actor frustrated with generic tools. Try it: tonetutor.tefusiang.com free for 3 sessions Curious about the speech-to-text pipeline or tone grading logic? Ask below.