{"slug": "i-built-a-full-image-editor-that-never-uploads-your-photo-here-s-how-the-on-ai", "title": "I built a full image editor that never uploads your photo — here's how the on-device AI parts actually work", "summary": "A developer built APIC-Web, a full image editor that runs entirely in the browser using Canvas, WebAssembly, and on-device machine learning, ensuring photos never leave the device. The project demonstrates how to achieve high-quality background removal by using confidence masks instead of hard labels, and implements exemplar-based texture synthesis for object removal, avoiding the need for server-side diffusion models.", "body_md": "Most background-removal and object-removal tools online work the same way: you upload an image, a server runs a model on it, you download the result. Convenient, but your photo left your device and you have no idea what happened to that copy afterward.\n\nI wanted to see how far I could push the \"everything stays client-side\" constraint for a full image editor — background removal, object removal, filters, format conversion, compression — using only what a browser can do natively: Canvas, WebAssembly, and an on-device ML model. This is APIC-Web, and here's what I learned building the two hardest parts.\n\nThe obvious approach is MediaPipe's ImageSegmenter with outputCategoryMask: true — it hands you a hard 0/1 label per pixel (background or foreground). It works, but the output has a visible jagged, stair-stepped edge around hair and shoulders. There's no way to fix that after the fact with a blur, because the information needed to know how much a boundary pixel belongs to the foreground is already gone — it got rounded to a hard integer.\n\nThe fix is outputConfidenceMasks: true instead, which returns a float 0–1 probability per pixel before it gets thresholded. That single change unlocks a proper edge pipeline:\n\njs\n\n// 1. Smoothstep contrast — push clear foreground/background toward pure\n\n// opaque/transparent while keeping the ~0.5 transition band soft\n\nconst s = v * v * (3 - 2 * v);\n\n// 2. Erode ~1px inward — kills background-colour fringing around hair\n\n// (a min-filter over a 3x3 neighbourhood)\n\n// 3. Feather with a small separable box blur — turns the eroded edge\n\n// into something smooth instead of jagged\n\nNet effect: same underlying model, dramatically better-looking edges, because you're working with the mask's actual uncertainty instead of throwing it away too early.\n\nOne gotcha worth flagging for anyone doing the same thing: not every segmentation model outputs the same number of confidence channels. Some give you [background, foreground](https://dev.to2%20channels), others give you a single foreground-probability channel. Don't hardcode confidenceMasks[1] — check the array length and handle both, and wrap model creation so a GPU delegate failure falls back to CPU instead of just dying.\n\nPhoto editors with \"magic eraser\" features (Photoshop's Generative Fill, Galaxy AI's object eraser) use trained diffusion models. That's not something you can ship for free in a static HTML page — no server, no GPU inference budget.\n\nWhat is achievable client-side is exemplar-based texture synthesis — essentially the pre-AI generation of content-aware fill. The idea:\n\nTake the painted (masked) region plus a padded area of real surrounding pixels\n\nBreak the hole into small blocks (8×8) and process them boundary-inward — the block with the most already-resolved neighbours goes first\n\nFor each block, search the surrounding valid texture for the patch that minimizes SSD against the parts of the target block that are already known\n\nCopy those real pixels in — not an average, not a blur, an actual texture copy — which is what keeps the result sharp instead of smeared\n\njs\n\nfor (let py = 0; py < bhei; py++) {\n\nfor (let px = 0; px < bwid; px++) {\n\nconst tIdx = (ty0 + py) * ww + (tx0 + px);\n\nif (!validSrc[tIdx]) continue; // only compare known-good texture\n\nconst cIdx = (cy + py) * ww + (cx + px);\n\nconst dr = wd[tIdx*4] - wd[cIdx*4];\n\n// ...accumulate squared error across candidate positions\n\n}\n\n}\n\nThe subtle bug I hit here: after background removal, transparent pixels still have leftover garbage RGB values sitting under alpha: 0 — they're just not painted, so a naive \"is this pixel painted?\" check treats them as valid source texture. The fix was splitting the concept into two separate arrays: isPainted (defines the hole) and validSrc (opaque and unpainted — the only pixels safe to copy from). Mixing those two concepts up is exactly how you get a random smear of background color pasted into your subject.\n\nPerformance-wise, brute-force patch search is O(hole_pixels × search_area), which gets ugly fast on a large image. Restricting the search to a padded bounding box around the mask (not the whole image) and adaptively adjusting the candidate sampling stride based on hole size keeps this bounded — typical fills run in well under a second.\n\nTakeaways\n\nIf your segmentation output has visible jagged edges, you're probably throwing away confidence information too early — switch to soft masks and add erosion + feather.\n\nYou don't need a trained model to get decent object removal; exemplar-based patch matching gets you most of the way for everyday use cases (skin, sky, fabric, walls), it's just an older, well-understood technique.\n\nWhen you're compositing filled/generated pixels back into a photo that's already been alpha-masked, be explicit about what counts as \"safe to sample from\" — transparency and \"not edited\" are not the same thing, and conflating them is a real bug, not a hypothetical one.\n\nAPIC-Web is live and free if you want to see the result:\n\n[https://akhouri-anmol-kumar.github.io/APIC-web/](https://akhouri-anmol-kumar.github.io/APIC-web/)\n\nFeedback — especially edge cases that break it — genuinely welcome.\n\n\"We Build What Others Forgot To Fix\"", "url": "https://wpnews.pro/news/i-built-a-full-image-editor-that-never-uploads-your-photo-here-s-how-the-on-ai", "canonical_source": "https://dev.to/akhourianmolkumar/i-built-a-full-image-editor-that-never-uploads-your-photo-heres-how-the-on-device-ai-parts-1i03", "published_at": "2026-08-24 04:14:07+00:00", "updated_at": "2026-08-24 04:43:46.631439+00:00", "lang": "en", "topics": ["machine-learning", "computer-vision", "developer-tools", "ai-products"], "entities": ["APIC-Web", "MediaPipe"], "alternates": {"html": "https://wpnews.pro/news/i-built-a-full-image-editor-that-never-uploads-your-photo-here-s-how-the-on-ai", "markdown": "https://wpnews.pro/news/i-built-a-full-image-editor-that-never-uploads-your-photo-here-s-how-the-on-ai.md", "text": "https://wpnews.pro/news/i-built-a-full-image-editor-that-never-uploads-your-photo-here-s-how-the-on-ai.txt", "jsonld": "https://wpnews.pro/news/i-built-a-full-image-editor-that-never-uploads-your-photo-here-s-how-the-on-ai.jsonld"}}