{"slug": "introducing-the-html-in-canvas-api-origin-trial", "title": "Introducing the HTML-in-Canvas API origin trial", "summary": "The article introduces the new experimental HTML-in-Canvas API, now available in an origin trial, which allows developers to render DOM content directly into a 2D canvas or WebGL/WebGPU texture while maintaining interactivity, accessibility, and browser feature integration. This API bridges the gap between the DOM's rich semantic features and the canvas's high-performance graphics, enabling text selection, find-in-page, accessibility support, and extension compatibility within canvas-based applications. It aims to solve the longstanding architectural trade-off for complex web apps like Google Docs or Figma, which previously had to choose between DOM functionality and low-level graphics performance.", "body_md": "For years, web developers have had to make a tough architectural choice when building complex, highly-interactive visual applications on the web: do you lean on the DOM for its rich semantic features, or do you render directly to the ``\n\nelement for low-level graphics performance?\n\nWith the new experimental [ HTML-in-Canvas API](https://github.com/WICG/html-in-canvas/)—available now\n\n[in origin trial](/origintrials#/view_trial/3478467762190286849)—you don't have to choose. This API lets you draw DOM content directly into a 2D canvas or a WebGL/WebGPU texture while keeping the UI interactable, accessible, and hooked up to your favorite browser features. By combining HTML with low-level graphics processing, you can create experiences that were previously impossible.\n\n## The DOM versus Canvas\n\nTo understand the power of this new API, it helps to look at the relative strengths of both the DOM and the Canvas.\n\n[The DOM](https://developer.mozilla.org/docs/Web/API/Document_Object_Model) is the staple of web UI. It offers text layout solutions out of the box, using semantically understood content to create rich interfaces. This lets users perform common operations across web pages seamlessly—things we often take for granted, like highlighting text to copy, or right-clicking an image to save it. The DOM also integrates with essential browser features: accessibility tools, translate, find-in-page, reader mode, extensions, dark mode, browser zoom, and autofill.\n\n[Canvas](https://developer.mozilla.org/docs/Web/API/Canvas_API) (and [WebGL](https://developer.mozilla.org/docs/Web/API/WebGL_API)/[WebGPU](https://developer.mozilla.org/docs/Web/API/WebGPU_API)), on the other hand, allows for low-level access to drive a grid of pixels for highly advanced 2D and 3D graphics. Games and complex web apps (like Google Docs or Figma) require this performant, low-level access. Because the canvas is fundamentally a grid of pixels, supporting features like responsive text used to require complex custom UI logic, drastically increasing your bundle size. Crucially, all the powerful browser features integrated into the DOM break completely when the UI is trapped inside a static canvas pixel grid.\n\n## The advantages of bringing the DOM to Canvas\n\nThe HTML-in-Canvas API is the bridge that gives you the best of both worlds. By placing HTML inside the ``\n\nelement and synchronizing its transform, you ensure the content remains fully interactive, and that all browser integrations function automatically.\n\nHere's what you get by letting the DOM handle your UI inside a ``\n\nelement:\n\n**Text layout and formatting:** Simplified text layout and formatting, including multiline or bidirectional text with CSS styles applied.**Form controls:** Expressive and easier to use form controls with extensive customization options.**Text selection, copy/paste, and right-click:** Users can highlight text inside your 3D scenes, or right-click context menus natively.**Text selection, copy/paste, and right-click:** Users can highlight text inside your 3D scenes, or right-click context menus natively.**Accessibility:** Content rendered inside the canvas is exposed to the accessibility tree. Accessibility systems can parse the UI as they do normal HTML, and expose it to systems like screen readers.**Find-in-page:** Users can use find-in-page (`Ctrl`/` Cmd`+` F`) to search for text, and the browser will highlight it directly within your WebGL textures.**Find-in-page:** Users can use find-in-page (`Ctrl`/` Cmd`+` F`) to search for text, and the browser will highlight it directly within your WebGL textures.**Indexability and AI agent interfaceable:** Web crawlers and AI agents can seamlessly index and read the text rendered into your 2D and 3D scenes.**Extension integration:** Browser extensions work natively. For example, a text-replacement extension will automatically update the text rendered on your 3D meshes.**DevTools integration:** You can inspect your canvas content, including for WebGL/WebGPU UI elements directly in Chrome DevTools. Tweak a CSS style in the inspector, and watch it instantly update on the 3D texture!\n\n## High-level use cases\n\nThis API unlocks incredible potential across several domains:\n\n**Large canvas-based applications:** Heavyweight web apps like Google Docs, Miro, or Figma can now render complex application UI components natively into their canvas-driven workspaces, improving accessibility and reducing bundle weight.**3D scenes and games:** Marketing sites, immersive WebXR experiences, and web games can now place fully interactable web UI into 3D scenes—like a 3D book that uses real DOM text, or an in-game terminal that natively supports copying and pasting.\n\n## How to use the API\n\nUsing the API happens in three phases: Setting up your canvas, rendering into the canvas, and updating the CSS transform so the browser knows where the element physically sits on the screen.\n\n### Prerequisites\n\nThe HTML-in-Canvas API is in origin trial in Chrome 148 through 150. To test it on your site, use Chrome Canary 149 or later with the `chrome://flags/#canvas-draw-element`\n\nflag enabled. To enable the API for other users, register for [the Origin Trial](/origintrials#/view_trial/3478467762190286849).\n\n### Step 1: Basic Canvas setup\n\nFirst, add the `layoutsubtree`\n\nattribute to your ``\n\ntag. This makes the browser aware of the content nested inside the canvas, preparing it to be displayed inside the canvas, and exposing it to accessibility trees.\n\n```\n\n \n Name:
\n \n```\n\n#### Size the canvas grid\n\nTo avoid blurriness of the rendered content, make sure to size the canvas grid to match the device scale factor.\n\n``` js\nconst observer = new ResizeObserver(([entry]) => {\n const dpc = entry.devicePixelContentBoxSize;\n canvas.width = dpc ? dpc[0].inlineSize : Math.round(entry.contentRect.width * window.devicePixelRatio);\n canvas.height = dpc ? dpc[0].blockSize : Math.round(entry.contentRect.height * window.devicePixelRatio);\n});\n\nconst supportsDevicePixelContentBox =\n typeof ResizeObserverEntry !== 'undefined' &&\n 'devicePixelContentBoxSize' in ResizeObserverEntry.prototype;\nconst options = supportsDevicePixelContentBox ? { box: 'device-pixel-content-box' } : {};\nobserver.observe(canvas, options);\n```\n\n### Step 2: Rendering\n\nFor a 2D context, use the `drawElementImage`\n\nmethod. Do this inside the `paint`\n\nevent, which triggers whenever the element redraws—for example, during text highlighting or user input. It's crucial to update the element's CSS transform with the return value so interactivity continues to work.\n\n``` js\nconst ctx = document.getElementById('canvas').getContext('2d');\nconst form_element = document.getElementById('form_element');\nconst canvas = document.getElementById('canvas');\n\ncanvas.onpaint = () => {\n ctx.reset();\n\n // Draw the form element at x:0, y:0\n let transform = ctx.drawElementImage(form_element, 0, 0);\n\n // Use the transform returned later on...\n};\n```\n\n#### Render with WebGL\n\nFor WebGL, you use `texElementImage2D`\n\n. It functions similar to `texImage2D`\n\n, but takes the DOM element as the source.\n\n``` js\ncanvas.onpaint = () => {\n if (gl.texElementImage2D) {\n gl.texElementImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, form_element);\n }\n};\n```\n\n#### Render with WebGPU\n\nWebGPU uses the `copyElementImageToTexture`\n\nmethod on the device queue, analogous to `copyExternalImageToTexture`\n\n:\n\n``` js\ncanvas.onpaint = () => {\n root.device.queue.copyElementImageToTexture(\n valueElement,\n { texture: targetTexture }\n );\n};\n```\n\n### Step 3: Update the CSS transform\n\nNow that you've rendered the element into the canvas you will need to update the browser on where it is located. This ensures spatial synchronization between the canvas and the DOM's layout. This is important so that the browser can correctly map the event zone—such where exactly the user clicks or hovers—with where the element is rendered.\n\nFor the 2D context case, apply the transform returned by the rendering call to the `.style.transform property`\n\n:\n\n``` js\nconst ctx = document.getElementById('canvas').getContext('2d');\nconst form_element = document.getElementById('form_element');\nconst canvas = document.getElementById('canvas');\n\ncanvas.onpaint = () => {\n ctx.reset();\n // Draw the form element at x:0, y:0\n let transform = ctx.drawElementImage(form_element, 0, 0);\n\n // Sync the DOM location with the drawn location\n form_element.style.transform = transform.toString();\n};\n```\n\nWith WebGL or WebGPU, the on-screen location of an element depends on how the output texture is used by shader code, and can't be deduced from the canvas rendering context. However, if your shader program uses a typical model view projection to draw the texture, then you can use the new convenience function `element.getElementTransform()`\n\nto compute a transform that can be used in the same way as the return value from `drawElementImage()`\n\n. To facilitate this, you need to do the following:\n\n**Convert WebGL**[MVP Matrix](https://developer.mozilla.org/docs/Web/API/WebGL_API/WebGL_model_view_projection#the_model_view_and_projection_matrices)to[DOM Matrix](https://developer.mozilla.org/docs/Web/API/DOMMatrix).**Normalize the HTML element.** HTML elements are sized in pixels (for example, 200px wide). WebGL, however, usually treats objects as \"unit squares\", for example, ranging from 0 to 1. If you don't normalize, your 200px button will look 200 times larger.**Map to the canvas viewport.** This step is the \"rescaling\" phase: it stretches that unit-space math back out to match the actual pixel dimensions of your``\n\nelement on the screen. It also flips the Y-axis, because in WebGL, up is positive, but in CSS, down is positive.**Calculate the final transform.** Multiply the matrixes in order:`Viewport * MVP * Normalization.`\n\nCombining them into one final transform produces a \"map\" that tells the browser exactly where that HTML element layer should sit to align with the 3D drawing.**Apply the transform to the HTML element.** This moves the HTML element layer to sit directly on top of its rendered pixels. This ensures that when a user clicks a button or selects text, they're hitting the real HTML element.\n\n```\nif (canvas.getElementTransform) {\n // 1. Convert WebGL MVP Matrix to DOM Matrix\n const mvpDOM = new DOMMatrix(Array.from(htmlElementMVP));\n\n // 2. Normalize the HTML element (pixels -> 1x1 unit square)\n const width = targetHTMLElement.offsetWidth;\n const height = targetHTMLElement.offsetHeight;\n\n const cssToUnitSpace = new DOMMatrix()\n .scale(1 / width, -1 / height, 1) // Shrink to unit size and flip Y\n .translate(-width / 2, -height / 2); // Center the element\n\n // 3. Map to the canvas viewport\n const clipToCanvasViewport = new DOMMatrix()\n .translate(canvas.width / 2, canvas.height / 2) // Move origin to center\n .scale(canvas.width / 2, -canvas.height / 2, 1); // Stretch to canvas dimensions\n\n // 4. Multiply: (Clip -> Pixels) * (MVP) * (pixels -> unit square)\n const screenSpaceTransform = clipToCanvasViewport\n .multiply(mvpDOM)\n .multiply(cssToUnitSpace);\n\n // 5. Apply to the transform\n const computedTransform = canvas.getElementTransform(targetHTMLElement, screenSpaceTransform);\n if (computedTransform) {\n targetHTMLElement.style.transform = computedTransform.toString();\n }\n}\n```\n\n## Library and framework support\n\nSome of the popular libraries have already shipped support for the HTML-in-Canvas feature.\n\n### Three.js\n\nUpdating matrixes manually can be tedious, which is why frameworks are already jumping on board. Three.js has [experimental support](https://github.com/mrdoob/three.js/pull/31233) using the new `THREE.HTMLTexture`\n\n:\n\n``` js\nconst material = new THREE.MeshBasicMaterial();\nmaterial.map = new THREE.HTMLTexture(uiElement); // Pass the DOM element\n\nconst geometry = new THREE.BoxGeometry(1, 1, 1);\nconst mesh = new THREE.Mesh(geometry, material);\nscene.add(mesh);\n```\n\n### PlayCanvas\n\nPlayCanvas also supports HTML-in-Canvas using their texture API:\n\n```\n// Wait for the 'paint' event to set the source\ncanvas.addEventListener('paint', () => {\n htmlTexture.setSource(htmlElement);\n}, { once: true });\ncanvas.requestPaint();\n\n// Keep up to date\ncanvas.addEventListener('paint', onPaintUpload);\n\nconst material = new pc.StandardMaterial();\nmaterial.diffuseMap = htmlTexture;\nmaterial.update();\n```\n\n## Demos\n\nBefore trying out the demos, ensure your environment is [properly configured](#prerequisites).\n\nThere are [several demos](https://github.com/WICG/html-in-canvas/tree/main/Examples) that serve as a reference for using the API. We are already seeing creative solutions from the community, ranging from translatable 3D books to UI elements that refract through glass shaders:\n\n[The 3D book](https://chrome.dev/html-in-canvas/demos/webgl-book-curl.html): A WebGL-rendered 3D book that uses HTML layout for its pages. Users can swap fonts with CSS. Because it's DOM-based, built-in translation works instantly, and AI agents can extract the text with less complexity.[Interactive 3D UIs](https://wicg.github.io/html-in-canvas/Examples/webgpu-jelly-slider/): A WebGPU jelly slider that refracts light based on an underlying 3D model, while still responding to standard HTML`