Built an AI that explains math visually instead of just answering

Talk to it. Watch it teach.

Claw Learn is an AI-powered visual math tutor with a real-time voice interface — powered by the ElevenLabs Speech Engine. Ask any math or physics question by voice or text, and watch a synchronized animated explanation generate live in the browser.

Claw Learn combines the ElevenLabs Speech Engine with an AI scene planner and a custom canvas renderer to turn math questions into live animated explanations with synchronized narration.

The Speech Engine is the core of the experience — it handles both voice input and audio output over WebRTC, so you can speak your question, interrupt mid-explanation, and ask follow-ups without ever touching a keyboard. When the Speech Engine isn't configured, the app falls back to REST TTS and browser-based speech recognition.

No slides. No textbooks. No pre-recorded videos. Every explanation is generated fresh for your exact question.

You:  "Why does the derivative represent slope?"

App:  → ElevenLabs Speech Engine captures your voice over WebRTC
      → AI generates a 10-scene visual teaching plan
      → Canvas renders: axes, parabola, tangent line, slope formula
      → Speech Engine narrates each scene in sync with the animation
      → Interrupt at any time to ask a follow-up — just speak

Add a GIF or screenshot here

Try these questions:

"How does matrix multiplication work?""Explain the Fourier transform visually""What is integration and why does it find area?""Show me Euler's formula e^(iπ) + 1 = 0""How does gravity create orbits?"

Layer	Technology
Framework	Next.js 16 (App Router, Turbopack)
UI	React 19, Tailwind CSS v4, Framer Motion
AI	Any OpenAI-compatible API (Gemini, OpenAI, Ollama, etc.)
Voice I/O	ElevenLabs Speech Engine (WebRTC)
TTS fallback	ElevenLabs REST API
STT fallback	Web Speech API
Animations	Custom 2D Canvas renderer
Language	TypeScript 5
Deployment	Vercel

Node.js 18+An OpenAI-compatible API key— Gemini (free ataistudio.google.com), OpenAI, or any compatible provider Google OAuth credentials— required for login (console.cloud.google.com)** Upstash Redis**— recommended for rate limiting (console.upstash.com, free tier)** ElevenLabs**— optional, free tier atelevenlabs.io

git clone https://github.com/arzumanabbasov/claw-learn.git
cd claw-learn
npm install
cp .env.local.example .env.local

Open .env.local

and fill in your keys:

OPENAI_API_KEY=your_api_key_here
OPENAI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai
OPENAI_MODEL=gemini-2.5-flash

AUTH_SECRET=your_auth_secret_here

GOOGLE_CLIENT_ID=your_google_client_id
GOOGLE_CLIENT_SECRET=your_google_client_secret

UPSTASH_REDIS_REST_URL=https://your-db.upstash.io
UPSTASH_REDIS_REST_TOKEN=your_token_here

ELEVENLABS_API_KEY=your_elevenlabs_api_key_here
ELEVENLABS_VOICE_ID=pNInz6obpgDQGcFmaJgB

ELEVENLABS_SPEECH_ENGINE_ID=agent_xxxxxxxxxxxxxxxxxxxx

ALLOWED_ORIGIN=https://your-domain.com
npm run dev

Open http://localhost:3000.

Claw Learn uses NextAuth.js v5 with Google OAuth. All routes require a valid session — unauthenticated users are redirected to /login

.

Setup:

• Go to console.cloud.google.com→ APIs & Services → Credentials - Create an OAuth 2.0 Client ID (Web application)
• Add your domain to Authorized JavaScript origins andhttps://your-domain.com/api/auth/callback/google

toAuthorized redirect URIs - Copy the Client ID and Secret into your env vars

• Generate AUTH_SECRET

withopenssl rand -base64 32

For local dev, add http://localhost:3000

as an authorized origin and http://localhost:3000/api/auth/callback/google

as a redirect URI.

Each authenticated user gets 3 questions per day, tracked by their Google user ID and reset at UTC midnight.

Rate limiting uses Upstash Redis in production — an atomic INCR

with a TTL set to the end of the current UTC day. This is serverless-safe and works across all Vercel edge instances.

Without Upstash credentials, the app falls back to an in-memory store that resets whenever the server restarts (fine for local dev, not suitable for production).

Setup:

• Create a free Redis database at console.upstash.com - Copy the REST URL and token into UPSTASH_REDIS_REST_URL

andUPSTASH_REDIS_REST_TOKEN

The remaining question count is shown in the top bar as a live badge and resets automatically each day.

Claw Learn uses the OpenAI-compatible API format, so it works with any provider that supports it.

OPENAI_API_KEY=your_gemini_api_key
OPENAI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai
OPENAI_MODEL=gemini-2.5-flash
OPENAI_API_KEY=your_openai_api_key
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_MODEL=gpt-4o
OPENAI_API_KEY=ollama
OPENAI_BASE_URL=http://localhost:11434/v1
OPENAI_MODEL=llama3.1

The Speech Engine connects via WebRTC to an ElevenLabs Conversational AI agent and is the primary voice interface for Claw Learn. It handles both input and output in a single low-latency connection:

Voice input— speak your questions naturally, no typing needed** Streaming TTS**— audio streams directly from ElevenLabs as each scene plays** Interruption**— speak mid-explanation to redirect or ask a follow-up** Lower latency**— WebRTC is significantly faster than the REST fallback

Setup:

• Go to elevenlabs.io/app/conversational-ai - Create a new agent
• Set the system prompt to: "You are a math narration voice. Read exactly what the user sends you as clear, natural narration." - Copy the Agent ID and set it as ELEVENLABS_SPEECH_ENGINE_ID

in your.env.local

The Voice button in the top bar connects and disconnects the Speech Engine. When connected, a pulsing green indicator shows the session is live.

When ELEVENLABS_API_KEY

is set but no Speech Engine is configured, each scene's narration is sent to the ElevenLabs REST API and played back as audio. No voice input in this mode.

The app works fully without any ElevenLabs configuration — text input and silent animations only.

npx vercel

Set these environment variables in the Vercel dashboard under Settings → Environment Variables:

Variable	Required	Description
OPENAI_API_KEY
✅	API key for your AI provider
OPENAI_BASE_URL
✅	Base URL of the OpenAI-compatible endpoint
OPENAI_MODEL
✅	Model name to use
AUTH_SECRET
✅	NextAuth secret (openssl rand -base64 32 )
GOOGLE_CLIENT_ID
✅	Google OAuth client ID
GOOGLE_CLIENT_SECRET
✅	Google OAuth client secret
UPSTASH_REDIS_REST_URL
Recommended	Upstash Redis URL for persistent rate limiting
UPSTASH_REDIS_REST_TOKEN
Recommended	Upstash Redis token
ELEVENLABS_API_KEY
Optional	ElevenLabs REST TTS fallback
ELEVENLABS_VOICE_ID
Optional	Override default voice
ELEVENLABS_SPEECH_ENGINE_ID
Recommended	WebRTC voice agent ID
ALLOWED_ORIGIN
Recommended	Your production domain for CORS

The vercel.json

in the repo is pre-configured.

npm run build
npm start

Requires Node.js 18+ and the environment variables above.

The canvas renderer supports 30+ element types:

Type	Description
axes
Coordinate axes with grid and tick labels
graph
Function curve (JS math expression)
tangent
Tangent line to a curve at a point
secant
Secant line between two points
shaded_area
Filled area under a curve
point
Dot with optional label
vector
Arrow with label
matrix
Matrix grid with brackets and highlights
formula
Math text in a pill box
histogram
Bar chart for distributions
pie_chart
Proportions and compositions
bar_chart
Categorical comparisons
line_chart
Discrete data series
scatter_plot
Correlation with optional regression line
wave
Propagating sine/cosine wave
axes_3d
Isometric 3D axes
complex_plane
Re/Im axes with unit circle
riemann_sum
Rectangles approximating an integral
slope_field
Directional arrows for dy/dx
parametric_curve
x(t), y(t) traced as t varies
polygon
Arbitrary shape from vertices
angle_arc
Label an angle between two rays
spring
Physics spring between two points
brace
Curly brace annotation
table
Data table with headers
highlight_region
Shaded overlay

Coordinate system: origin at center, x right, y up. Typical visible range: x ∈ [-6, 6], y ∈ [-4, 4].

clawlearn/
├── app/
│   ├── api/
│   │   ├── explain/route.ts          # POST — AI scene plan generation
│   │   ├── narrate/route.ts          # POST — ElevenLabs REST TTS
│   │   └── speech-engine/token/      # GET  — WebRTC conversation token
│   ├── page.tsx                      # Root — landing ↔ tutor router
│   ├── layout.tsx                    # Fonts, metadata, global CSS
│   └── globals.css                   # Design tokens, animations
│
├── components/
│   ├── LandingPage.tsx               # Marketing page
│   ├── TutorApp.tsx                  # App shell
│   ├── AnimationCanvas.tsx           # Canvas + scene sequencer
│   ├── ConversationPanel.tsx         # Chat history
│   ├── QuestionInput.tsx             # Input bar
│   └── NarrationSubtitle.tsx         # Subtitle below canvas
│
├── hooks/
│   ├── useTutor.ts                   # Core orchestration
│   ├── useSpeechEngine.ts            # ElevenLabs Speech Engine (WebRTC)
│   └── useVoice.ts                   # Web Speech API fallback
│
├── lib/
│   ├── openai.ts                     # OpenAI-compatible client + system prompt
│   ├── animationEngine.ts            # Canvas renderer (30+ elements)
│   ├── elevenlabs.ts                 # ElevenLabs REST helpers
│   └── voiceRecognition.ts           # Web Speech API wrapper
│
├── types/
│   └── scene.ts                      # Scene plan TypeScript types
│
├── .env.local.example                # Environment variable template
├── CONTRIBUTING.md                   # Contribution guide
├── LICENSE                           # MIT
└── vercel.json                       # Vercel deployment config

• API keys are server-side only — never exposed to the browser
• Input is length-limited and validated on every API route
• CORS is locked to ALLOWED_ORIGIN

in production - The canvas renderer uses a safe recursive-descent math parser — no eval

ornew Function

• Security headers ( X-Frame-Options

,X-Content-Type-Options

,Referrer-Policy

,Permissions-Policy

) are set on all responses

See SECURITY.md for the full security policy and how to report vulnerabilities.

No persistence— conversation history is in-memory, cleared on page refresh** Voice input**— Web Speech API fallback requires Chrome or Edge; the Speech Engine works in all modern browsers** JSON truncation**— very complex topics may cause the AI to return truncated JSON; the parser attempts recovery by finding the last complete sceneElevenLabs free tier— 10,000 characters/month; the app continues silently without narration when quota is exceeded

Contributions are welcome. Please read CONTRIBUTING.md first.

git checkout -b feat/your-feature
npx tsc --noEmit   # must pass
git commit -m "feat: your feature"
git push origin feat/your-feature

• Inspired by 3Blue1Brownand themanimanimation library - Built with the ElevenLabs Speech Engine,Next.js, andFramer Motion

MIT © 2025 Claw Learn Contributors

source & further reading

github.com — original article