# DIY AI Physical Therapist: Real-Time Pose Correction with React Native and MediaPipe

> Source: <https://dev.to/beck_moulton/diy-ai-physical-therapist-real-time-pose-correction-with-react-native-and-mediapipe-gbo>
> Published: 2026-09-08 00:05:00+00:00

Have you ever tried doing physical therapy exercises at home, only to wonder if your form is actually helping or just making things worse? 🤕 Traditional home rehabilitation often lacks the "watchful eye" of a professional. However, with the rise of **on-device computer vision** and **React Native development**, we can now build powerful, low-latency movement correction tools that run directly on a smartphone. 

In this tutorial, we are diving deep into **MediaPipe pose estimation** and **mobile AI integration** to build a "Smart Rehab Coach." We’ll explore how to capture real-time landmarks, calculate joint angles, and provide instant feedback—all while maintaining 60 FPS performance. By leveraging **TensorFlow Lite** and on-device processing, we ensure user privacy while delivering a seamless experience. 🚀

To achieve real-time feedback, we need a pipeline that minimizes the "bridge" overhead in React Native. We use a frame processor to pipe camera data directly into the MediaPipe inference engine.

``` php
graph TD
  A[Camera Stream] -->|Native Frame| B(MediaPipe Pose Landmarker)
  B -->|33 3D Landmarks| C{Angle Calculation Engine}
  C -->|Compare| D[Reference Library]
  D -->|Feedback| E[React Native UI Overlay]
  E -->|Visual Cues| F[User]

  subgraph "On-Device Processing"
  B
  C
  D
  end
```

Before we start coding, ensure your environment is ready:

The heart of our application is the MediaPipe Pose Landmarker. Unlike cloud-based solutions, this runs locally on the device's NPU/GPU.

``` js
import { PoseLandmarker, FilesetResolver } from "@mediapipe/tasks-vision";

async function createPoseLandmarker() {
  const vision = await FilesetResolver.forVisionTasks(
    "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@latest/wasm"
  );

  const poseLandmarker = await PoseLandmarker.createFromOptions(vision, {
    baseOptions: {
      modelAssetPath: `pose_landmarker_lite.task`,
      delegate: "GPU" // Critical for real-time performance ⚡
    },
    runningMode: "VIDEO",
    numPoses: 1
  });

  return poseLandmarker;
}
```

To tell a user their arm isn't straight enough, we need to calculate the angle between three points (e.g., Shoulder, Elbow, Wrist). We use the **Law of Cosines** or the `atan2` function.

``` js
// Helper to calculate angle between three landmarks
const calculateAngle = (a: Landmark, b: Landmark, c: Landmark): number => {
  const radians = Math.atan2(c.y - b.y, c.x - b.x) - 
                  Math.atan2(a.y - b.y, a.x - b.x);
  let angle = Math.abs((radians * 180.0) / Math.PI);

  if (angle > 180.0) {
    angle = 360 - angle;
  }
  return angle;
};

// Implementation for a Bicep Curl or Overhead Press
const checkForm = (landmarks: Landmark[]) => {
  const shoulder = landmarks[11];
  const elbow = landmarks[13];
  const wrist = landmarks[15];

  const elbowAngle = calculateAngle(shoulder, elbow, wrist);

  if (elbowAngle > 160) return "Fully Extended";
  if (elbowAngle < 45) return "Good Contraction";
  return "Keep Moving!";
};
```

In React Native, we use `react-native-vision-camera` with a frame processor. This allows us to run our logic on every single frame captured by the lens.

``` js
function RehabCamera() {
  const frameProcessor = useFrameProcessor((frame) => {
    'worklet';
    const poses = detectPose(frame); // Native call to MediaPipe

    if (poses.length > 0) {
      const angle = calculateAngle(poses[0].landmarks[11], ...);

      // Update UI via Shared Values
      if (angle < 90) {
        feedbackText.value = "Lower your hips!";
      }
    }
  }, []);

  return (
    <Camera
      device={device}
      isActive={true}
      frameProcessor={frameProcessor}
    />
  );
}
```

Building a production-ready vision app involves more than just landmark detection. You need to handle jitter (using a Kalman Filter), varying light conditions, and different body types.

For those looking to scale this into an enterprise-grade solution, check out the specialized patterns on **[WellAlly Tech Blog](https://www.wellally.tech/blog)**. They cover advanced topics like:

Integrating these patterns ensures that your app doesn't just "detect" poses, but actually "understands" human movement at a clinical level.

On-device AI is transforming how we approach healthcare and fitness. By combining **React Native** for the UI and **MediaPipe** for the intelligence, we can create low-latency, private, and highly effective rehabilitation tools.

The future of physical therapy isn't just in the clinic—it's in the pocket of every patient. 📱💪

**Are you building something with Pose Estimation? Drop a comment below or share your repo! I’d love to see how you’re handling landmark smoothing!**