Building a Browser-Based ID Photo Maker: Face Detection, Head Estimation & Background Removal

A developer built a fully client-side ID photo maker using a three-tier face detection chain (MediaPipe, Browser FaceDetector, BlazeFace) with anthropometric head measurement and background removal. The solution runs entirely in the browser, avoiding server costs and enabling instant preview.

The browser has become a surprisingly capable platform for computer vision. In this post, I'll walk through building a fully client-side ID photo maker — covering multi-tier face detection, anthropometric head measurement, segmentation-refined cropping, and background removal with color decontamination. ID photos have strict requirements: head height ratios, eye-line positions, margin rules. Tools like HivisionIDPhotos https://github.com/Zeyi-Lin/HivisionIDPhotos , dpar39/ppp https://github.com/dpar39/ppp , and commercial services like dreamega.ai https://dreamega.ai and photoaid.com https://photoaid.com all need to solve the same core problem: given an uploaded portrait, detect the face accurately enough to auto-crop to spec. Server-side tools can afford heavyweight models HivisionIDPhotos uses ONNX runtime with dedicated face parsing networks . But if you want to run everything client-side — no upload, no server cost, instant preview — the browser environment is unpredictable. Not every user has Chrome's experimental FaceDetector API, and WebGL support varies. Our solution: a three-tier detection chain that tries the best option first and gracefully falls back. // Detection chain: MediaPipe 478 landmarks → Browser FaceDetector → BlazeFace export async function detectFace image: HTMLImageElement : Promise<FaceDetectResult { if typeof window == "undefined" && window.isSecureContext { return { supported: false, face: null, reason: "insecure-context" }; } // 1. MediaPipe Face Landmarker – best quality, 478 landmarks try { const mpResult = await detectWithMediaPipe image ; if mpResult.face || mpResult.reason === "no-face-found" { return mpResult; } } catch { / Fall through / } // 2. Browser FaceDetector API Chrome const detector = await getDetector ; if detector { try { const nativeResult = await detectWithNative detector, image ; if nativeResult.face || nativeResult.reason === "no-face-found" { return nativeResult; } } catch { / Fall through / } } // 3. BlazeFace TensorFlow model try { const fallbackResult = await detectWithBlazeFace image ; if fallbackResult.supported || fallbackResult.reason === "no-face-found" { return fallbackResult; } } catch { / Fall through / } return { supported: false, face: null, reason: "api-missing" }; } Each tier returns a FaceBox with bounding coordinates and optional landmarks eyes, forehead, chin . The key insight: each detector gives you different landmark richness , so the geometry estimation adapts. Accurate head measurement is crucial — a few pixels off and the photo fails compliance. The approach here borrows from dpar39/ppp https://github.com/dpar39/ppp 's CrownChinEstimator and extends it with a MediaPipe-first path: export function estimateHeadHeight face: FaceBox : number { // Best: MediaPipe forehead-to-chin + hair offset 1.35x multiplier // HivisionIDPhotos avoids this problem by using alpha-channel contour // detection after background removal — a neat trick if you have a server. if face.foreheadTop && face.chinBottom { const chinY = Math.max face.chinBottom.y, face.y + face.height ; const foreheadToChin = chinY - face.foreheadTop.y; return foreheadToChin 1.35; } // Good: ppp formula chinCrownCoeff = 1.7699 if face.leftEye && face.rightEye { const ipd = Math.hypot face.rightEye.x - face.leftEye.x, face.rightEye.y - face.leftEye.y ; const eyeMidY = face.leftEye.y + face.rightEye.y / 2; const chinY = face.y + face.height; const frownToChin = chinY - eyeMidY; return 1.77 ipd + frownToChin ; } // Fallback: anthropometric multiplier Farkas LG reference return face.height 1.35; } The 1.35x multiplier accounts for hair volume above the forehead — MediaPipe landmarks don't detect hair, so this is intentionally an overestimate to prevent head cropping. The IPD formula comes from anthropometric research Farkas LG, "Anthropometry of the Head and Face" . Landmarks estimate where the head top should be , but segmentation gives you pixel-accurate results. This is where combining both outperforms services like passport-photo.online https://passport-photo.online/ that rely purely on face detection — especially for people with voluminous hair: export function refineWithSegmentation geom: HeadGeometry, segBounds: { topY: number; bottomY: number } | null : HeadGeometry { if segBounds return geom; // Only use segmentation topY — bottomY detects body silhouette, not chin const refinedTopY = Math.min segBounds.topY, geom.topY ; return { ...geom, topY: refinedTopY, headHeight: geom.bottomY - refinedTopY, }; } An important gotcha: segmentation bottomY is the body bottom shoulders, torso , not the chin. Using it would over-extend the head measurement. We only take topY from segmentation hair boundary and keep the chin position from face landmarks. With accurate head geometry, auto-cropping becomes a three-step process: export function computeAutoFitCropState { imageWidth, imageHeight, face, spec, canvasWidth, canvasHeight, segBounds } { const rules = getNormalizedHeadRules spec ; const head = refineWithSegmentation estimateHeadGeometry face , segBounds ; // 1. SCALE — head matches target height ratio const targetH = guide.headBottom - guide.headTop; const scale = clampedTarget / Math.max 1, head.headHeight ; // 2. PLACE — two anchor strategies if rules.hasEyeLineRule && hasEyes { // Strategy A: eyes at target Y used by ICAO/US passport specs crop = cropFromAnchor eyeImg, { x: cw/2, y: eyeTargetY }, ... ; } else { // Strategy B: head top at target margin common in Asian passport specs crop = cropFromAnchor { x: head.centerX, y: head.topY }, ... ; } // 3. GUARD — prevent head from being cropped crop = guard crop, head, face, ... ; return crop; } The guard pass handles edge cases: crown too close to the top, chin below frame, or both overflowing in which case it scales down to fit . When replacing backgrounds, edge pixels bleed the original background color. We use @imgly/background-removal ISNet model — the same engine behind remove.bg https://remove.bg 's open-source alternative — then apply color decontamination: js // Decontaminate semi-transparent edge pixels for let i = 0; i < rPx.length; i += 4 { let a = rPx i + 3 / 255; // Erode very low alpha pixels — eliminates faint halo if a < 0.15 { rPx i + 3 = 0; continue; } // Squeeze alpha: remap 0.15, 1 → 0, 1 a = Math.min 1, a - 0.15 / 0.85 ; rPx i + 3 = Math.round a 255 ; // Color decontamination for edge pixels if a 0.01 && a < 0.95 { const inv = 1 - a; rPx i = clamp255 rPx i - inv bgR / a ; rPx i + 1 = clamp255 rPx i + 1 - inv bgG / a ; rPx i + 2 = clamp255 rPx i + 2 - inv bgB / a ; } } The math reverses the alpha compositing formula: if the composited color is C = α·F + 1-α ·B , then the true foreground is F = C - 1-α ·B / α . The alpha squeeze 0.15, 1 → 0, 1 acts as an erosion filter that kills the soft halo around hair. Progressive enhancement works for ML : The three-tier face detection chain means every user gets the best experience their browser supports, with no manual feature-flag management. Separate detection from estimation : Face detection gives you coordinates; head geometry estimation interprets them. Keeping these separate lets you swap in better models without touching the layout logic. Segmentation complements, doesn't replace landmarks : Use segmentation for what it's good at hair boundaries, background separation and landmarks for what they're good at facial feature positions . Mixing their strengths produces better results than either alone. Color decontamination matters : Without the alpha-unpremultiply step, every background replacement shows a visible color fringe. The three lines of math are worth it. Browser ML is production-ready : Between MediaPipe, TensorFlow.js, and @imgly https://dev.to/imgly /background-removal, you can build a full ID photo pipeline that runs entirely on the client — no server round-trips, no privacy concerns. Hope this helps if you're building similar browser-based CV tools. Happy to answer questions in the comments.