{"slug": "built-an-ai-that-explains-math-visually-instead-of-just-answering", "title": "Built an AI that explains math visually instead of just answering", "summary": "Claw Learn, an AI-powered visual math tutor, now generates live animated explanations with synchronized narration in response to voice or text questions. The tool combines the ElevenLabs Speech Engine with an AI scene planner and custom canvas renderer to create fresh, interactive tutorials for math and physics concepts without relying on slides or pre-recorded videos.", "body_md": "**Talk to it. Watch it teach.**\n\nClaw Learn is an AI-powered visual math tutor with a real-time voice interface — powered by the ElevenLabs Speech Engine. Ask any math or physics question by voice or text, and watch a synchronized animated explanation generate live in the browser.\n\nClaw Learn combines the ElevenLabs Speech Engine with an AI scene planner and a custom canvas renderer to turn math questions into live animated explanations with synchronized narration.\n\nThe Speech Engine is the core of the experience — it handles both voice input and audio output over WebRTC, so you can speak your question, interrupt mid-explanation, and ask follow-ups without ever touching a keyboard. When the Speech Engine isn't configured, the app falls back to REST TTS and browser-based speech recognition.\n\nNo slides. No textbooks. No pre-recorded videos. Every explanation is generated fresh for your exact question.\n\n```\nYou:  \"Why does the derivative represent slope?\"\n\nApp:  → ElevenLabs Speech Engine captures your voice over WebRTC\n      → AI generates a 10-scene visual teaching plan\n      → Canvas renders: axes, parabola, tangent line, slope formula\n      → Speech Engine narrates each scene in sync with the animation\n      → Interrupt at any time to ask a follow-up — just speak\n```\n\nAdd a GIF or screenshot here\n\n**Try these questions:**\n\n*\"How does matrix multiplication work?\"**\"Explain the Fourier transform visually\"**\"What is integration and why does it find area?\"**\"Show me Euler's formula e^(iπ) + 1 = 0\"**\"How does gravity create orbits?\"*\n\n| Layer | Technology |\n|---|---|\n| Framework | Next.js 16 (App Router, Turbopack) |\n| UI | React 19, Tailwind CSS v4, Framer Motion |\n| AI | Any OpenAI-compatible API (Gemini, OpenAI, Ollama, etc.) |\n| Voice I/O | ElevenLabs Speech Engine (WebRTC) |\n| TTS fallback | ElevenLabs REST API |\n| STT fallback | Web Speech API |\n| Animations | Custom 2D Canvas renderer |\n| Language | TypeScript 5 |\n| Deployment | Vercel |\n\n**Node.js 18+****An OpenAI-compatible API key**— Gemini (free at[aistudio.google.com](https://aistudio.google.com/app/apikey)), OpenAI, or any compatible provider** Google OAuth credentials**— required for login ([console.cloud.google.com](https://console.cloud.google.com/))** Upstash Redis**— recommended for rate limiting ([console.upstash.com](https://console.upstash.com/), free tier)** ElevenLabs**— optional, free tier at[elevenlabs.io](https://elevenlabs.io)\n\n```\ngit clone https://github.com/arzumanabbasov/claw-learn.git\ncd claw-learn\nnpm install\ncp .env.local.example .env.local\n```\n\nOpen `.env.local`\n\nand fill in your keys:\n\n```\n# ── AI Provider (required) ────────────────────────────────────────────────────\nOPENAI_API_KEY=your_api_key_here\nOPENAI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai\nOPENAI_MODEL=gemini-2.5-flash\n\n# ── Auth (required) ───────────────────────────────────────────────────────────\n# Generate a secret: openssl rand -base64 32\nAUTH_SECRET=your_auth_secret_here\n\n# Google OAuth — https://console.cloud.google.com/\nGOOGLE_CLIENT_ID=your_google_client_id\nGOOGLE_CLIENT_SECRET=your_google_client_secret\n\n# ── Rate limiting — Upstash Redis (recommended) ───────────────────────────────\n# Without these, rate limiting falls back to in-memory (resets on server restart)\n# Create a free Redis DB at https://console.upstash.com/\nUPSTASH_REDIS_REST_URL=https://your-db.upstash.io\nUPSTASH_REDIS_REST_TOKEN=your_token_here\n\n# ── ElevenLabs Voice (optional) ───────────────────────────────────────────────\nELEVENLABS_API_KEY=your_elevenlabs_api_key_here\nELEVENLABS_VOICE_ID=pNInz6obpgDQGcFmaJgB\n\n# Speech Engine — full WebRTC voice I/O\n# Create an agent at https://elevenlabs.io/app/conversational-ai\nELEVENLABS_SPEECH_ENGINE_ID=agent_xxxxxxxxxxxxxxxxxxxx\n\n# ── Security ──────────────────────────────────────────────────────────────────\nALLOWED_ORIGIN=https://your-domain.com\nnpm run dev\n```\n\nOpen [http://localhost:3000](http://localhost:3000).\n\nClaw Learn uses [NextAuth.js v5](https://authjs.dev/) with Google OAuth. All routes require a valid session — unauthenticated users are redirected to `/login`\n\n.\n\n**Setup:**\n\n- Go to\n[console.cloud.google.com](https://console.cloud.google.com/)→ APIs & Services → Credentials - Create an OAuth 2.0 Client ID (Web application)\n- Add your domain to\n**Authorized JavaScript origins** and`https://your-domain.com/api/auth/callback/google`\n\nto**Authorized redirect URIs** - Copy the Client ID and Secret into your env vars\n- Generate\n`AUTH_SECRET`\n\nwith`openssl rand -base64 32`\n\nFor local dev, add `http://localhost:3000`\n\nas an authorized origin and `http://localhost:3000/api/auth/callback/google`\n\nas a redirect URI.\n\nEach authenticated user gets **3 questions per day**, tracked by their Google user ID and reset at UTC midnight.\n\nRate limiting uses **Upstash Redis** in production — an atomic `INCR`\n\nwith a TTL set to the end of the current UTC day. This is serverless-safe and works across all Vercel edge instances.\n\nWithout Upstash credentials, the app falls back to an in-memory store that resets whenever the server restarts (fine for local dev, not suitable for production).\n\n**Setup:**\n\n- Create a free Redis database at\n[console.upstash.com](https://console.upstash.com/) - Copy the REST URL and token into\n`UPSTASH_REDIS_REST_URL`\n\nand`UPSTASH_REDIS_REST_TOKEN`\n\nThe remaining question count is shown in the top bar as a live badge and resets automatically each day.\n\nClaw Learn uses the OpenAI-compatible API format, so it works with any provider that supports it.\n\n```\nOPENAI_API_KEY=your_gemini_api_key\nOPENAI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai\nOPENAI_MODEL=gemini-2.5-flash\nOPENAI_API_KEY=your_openai_api_key\nOPENAI_BASE_URL=https://api.openai.com/v1\nOPENAI_MODEL=gpt-4o\nOPENAI_API_KEY=ollama\nOPENAI_BASE_URL=http://localhost:11434/v1\nOPENAI_MODEL=llama3.1\n```\n\nThe Speech Engine connects via WebRTC to an ElevenLabs Conversational AI agent and is the primary voice interface for Claw Learn. It handles both input and output in a single low-latency connection:\n\n**Voice input**— speak your questions naturally, no typing needed** Streaming TTS**— audio streams directly from ElevenLabs as each scene plays** Interruption**— speak mid-explanation to redirect or ask a follow-up** Lower latency**— WebRTC is significantly faster than the REST fallback\n\n**Setup:**\n\n- Go to\n[elevenlabs.io/app/conversational-ai](https://elevenlabs.io/app/conversational-ai) - Create a new agent\n- Set the system prompt to:\n*\"You are a math narration voice. Read exactly what the user sends you as clear, natural narration.\"* - Copy the Agent ID and set it as\n`ELEVENLABS_SPEECH_ENGINE_ID`\n\nin your`.env.local`\n\nThe **Voice** button in the top bar connects and disconnects the Speech Engine. When connected, a pulsing green indicator shows the session is live.\n\nWhen `ELEVENLABS_API_KEY`\n\nis set but no Speech Engine is configured, each scene's narration is sent to the ElevenLabs REST API and played back as audio. No voice input in this mode.\n\nThe app works fully without any ElevenLabs configuration — text input and silent animations only.\n\n```\nnpx vercel\n```\n\nSet these environment variables in the Vercel dashboard under **Settings → Environment Variables**:\n\n| Variable | Required | Description |\n|---|---|---|\n`OPENAI_API_KEY` |\n✅ | API key for your AI provider |\n`OPENAI_BASE_URL` |\n✅ | Base URL of the OpenAI-compatible endpoint |\n`OPENAI_MODEL` |\n✅ | Model name to use |\n`AUTH_SECRET` |\n✅ | NextAuth secret (`openssl rand -base64 32` ) |\n`GOOGLE_CLIENT_ID` |\n✅ | Google OAuth client ID |\n`GOOGLE_CLIENT_SECRET` |\n✅ | Google OAuth client secret |\n`UPSTASH_REDIS_REST_URL` |\nRecommended | Upstash Redis URL for persistent rate limiting |\n`UPSTASH_REDIS_REST_TOKEN` |\nRecommended | Upstash Redis token |\n`ELEVENLABS_API_KEY` |\nOptional | ElevenLabs REST TTS fallback |\n`ELEVENLABS_VOICE_ID` |\nOptional | Override default voice |\n`ELEVENLABS_SPEECH_ENGINE_ID` |\nRecommended | WebRTC voice agent ID |\n`ALLOWED_ORIGIN` |\nRecommended | Your production domain for CORS |\n\nThe `vercel.json`\n\nin the repo is pre-configured.\n\n```\nnpm run build\nnpm start\n```\n\nRequires Node.js 18+ and the environment variables above.\n\nThe canvas renderer supports 30+ element types:\n\n| Type | Description |\n|---|---|\n`axes` |\nCoordinate axes with grid and tick labels |\n`graph` |\nFunction curve (JS math expression) |\n`tangent` |\nTangent line to a curve at a point |\n`secant` |\nSecant line between two points |\n`shaded_area` |\nFilled area under a curve |\n`point` |\nDot with optional label |\n`vector` |\nArrow with label |\n`matrix` |\nMatrix grid with brackets and highlights |\n`formula` |\nMath text in a pill box |\n`histogram` |\nBar chart for distributions |\n`pie_chart` |\nProportions and compositions |\n`bar_chart` |\nCategorical comparisons |\n`line_chart` |\nDiscrete data series |\n`scatter_plot` |\nCorrelation with optional regression line |\n`wave` |\nPropagating sine/cosine wave |\n`axes_3d` |\nIsometric 3D axes |\n`complex_plane` |\nRe/Im axes with unit circle |\n`riemann_sum` |\nRectangles approximating an integral |\n`slope_field` |\nDirectional arrows for dy/dx |\n`parametric_curve` |\nx(t), y(t) traced as t varies |\n`polygon` |\nArbitrary shape from vertices |\n`angle_arc` |\nLabel an angle between two rays |\n`spring` |\nPhysics spring between two points |\n`brace` |\nCurly brace annotation |\n`table` |\nData table with headers |\n`highlight_region` |\nShaded overlay |\n\n**Coordinate system:** origin at center, x right, y up. Typical visible range: x ∈ [-6, 6], y ∈ [-4, 4].\n\n```\nclawlearn/\n├── app/\n│   ├── api/\n│   │   ├── explain/route.ts          # POST — AI scene plan generation\n│   │   ├── narrate/route.ts          # POST — ElevenLabs REST TTS\n│   │   └── speech-engine/token/      # GET  — WebRTC conversation token\n│   ├── page.tsx                      # Root — landing ↔ tutor router\n│   ├── layout.tsx                    # Fonts, metadata, global CSS\n│   └── globals.css                   # Design tokens, animations\n│\n├── components/\n│   ├── LandingPage.tsx               # Marketing page\n│   ├── TutorApp.tsx                  # App shell\n│   ├── AnimationCanvas.tsx           # Canvas + scene sequencer\n│   ├── ConversationPanel.tsx         # Chat history\n│   ├── QuestionInput.tsx             # Input bar\n│   └── NarrationSubtitle.tsx         # Subtitle below canvas\n│\n├── hooks/\n│   ├── useTutor.ts                   # Core orchestration\n│   ├── useSpeechEngine.ts            # ElevenLabs Speech Engine (WebRTC)\n│   └── useVoice.ts                   # Web Speech API fallback\n│\n├── lib/\n│   ├── openai.ts                     # OpenAI-compatible client + system prompt\n│   ├── animationEngine.ts            # Canvas renderer (30+ elements)\n│   ├── elevenlabs.ts                 # ElevenLabs REST helpers\n│   └── voiceRecognition.ts           # Web Speech API wrapper\n│\n├── types/\n│   └── scene.ts                      # Scene plan TypeScript types\n│\n├── .env.local.example                # Environment variable template\n├── CONTRIBUTING.md                   # Contribution guide\n├── LICENSE                           # MIT\n└── vercel.json                       # Vercel deployment config\n```\n\n- API keys are server-side only — never exposed to the browser\n- Input is length-limited and validated on every API route\n- CORS is locked to\n`ALLOWED_ORIGIN`\n\nin production - The canvas renderer uses a safe recursive-descent math parser — no\n`eval`\n\nor`new Function`\n\n- Security headers (\n`X-Frame-Options`\n\n,`X-Content-Type-Options`\n\n,`Referrer-Policy`\n\n,`Permissions-Policy`\n\n) are set on all responses\n\nSee [SECURITY.md](/arzumanabbasov/claw-learn/blob/main/SECURITY.md) for the full security policy and how to report vulnerabilities.\n\n**No persistence**— conversation history is in-memory, cleared on page refresh** Voice input**— Web Speech API fallback requires Chrome or Edge; the Speech Engine works in all modern browsers** JSON truncation**— very complex topics may cause the AI to return truncated JSON; the parser attempts recovery by finding the last complete scene**ElevenLabs free tier**— 10,000 characters/month; the app continues silently without narration when quota is exceeded\n\nContributions are welcome. Please read [CONTRIBUTING.md](/arzumanabbasov/claw-learn/blob/main/CONTRIBUTING.md) first.\n\n```\n# Fork, then:\ngit checkout -b feat/your-feature\n# Make changes\nnpx tsc --noEmit   # must pass\ngit commit -m \"feat: your feature\"\ngit push origin feat/your-feature\n# Open a pull request\n```\n\n- Inspired by\n[3Blue1Brown](https://www.3blue1brown.com/)and the[manim](https://github.com/3b1b/manim)animation library - Built with the\n[ElevenLabs Speech Engine](https://elevenlabs.io/conversational-ai),[Next.js](https://nextjs.org/), and[Framer Motion](https://www.framer.com/motion/)\n\n[MIT](/arzumanabbasov/claw-learn/blob/main/LICENSE) © 2025 Claw Learn Contributors", "url": "https://wpnews.pro/news/built-an-ai-that-explains-math-visually-instead-of-just-answering", "canonical_source": "https://github.com/arzumanabbasov/claw-learn", "published_at": "2026-05-29 20:26:26+00:00", "updated_at": "2026-05-29 20:47:56.342563+00:00", "lang": "en", "topics": ["ai-products", "ai-tools", "generative-ai", "natural-language-processing", "ai-startups"], "entities": ["Claw Learn", "ElevenLabs", "ElevenLabs Speech Engine", "WebRTC", "REST TTS"], "alternates": {"html": "https://wpnews.pro/news/built-an-ai-that-explains-math-visually-instead-of-just-answering", "markdown": "https://wpnews.pro/news/built-an-ai-that-explains-math-visually-instead-of-just-answering.md", "text": "https://wpnews.pro/news/built-an-ai-that-explains-math-visually-instead-of-just-answering.txt", "jsonld": "https://wpnews.pro/news/built-an-ai-that-explains-math-visually-instead-of-just-answering.jsonld"}}