# We made Bryntum Gantt look handmade with realtime image diffusion

> Source: <https://bryntum.com/blog/we-made-bryntum-gantt-look-handmade-with-realtime-image-diffusion/>
> Published: 2026-09-11 10:44:39+00:00

# We made Bryntum Gantt look handmade with realtime image diffusion

We strive to keep posts updated, but code samples may sometimes be outdated. Humans, see the [Bryntum documentation](https://bryntum.com/docs/); agents, [https://mcp.bryntum.com](https://mcp.bryntum.com) for the latest info.

We found an artisanal [cattery booking system](https://www.reddit.com/r/mildlyinteresting/comments/1vqloem/the_uncomputerized_booking_system_at_my_cats/) on Reddit. Its handwritten labels and physical markers made us wonder: can we make our [Bryntum Gantt chart](https://bryntum.com/products/gantt/) have a handmade look?

Further inspiration from this [Syntax.fm YouTube short](https://www.youtube.com/shorts/U7peCdBiFy8) made us realize we can style it however we want in real time using only prompts:

You can try the demo yourself by cloning the [Bryntum Gantt with realtime image diffusion demo GitHub repo](https://github.com/bryntum/bryntum-gantt-realtime-image-diffusion-demo), installing dependencies with `npm install`, and starting the dev server with `npm run dev`. It uses Vite, React, and the Bryntum Gantt trial package, so no license is needed to run it. Generating the styled frames requires signing up for the generative AI platform [fal](https://fal.ai/) for an API key. At the time of writing you get $5 free credits, which is enough to try it out. The demo also requires Chrome Canary or Brave, as it uses the experimental [HTML-in-Canvas Web API](https://developer.chrome.com/blog/html-in-canvas-origin-trial) with two browser flags enabled, as explained in the demo repo README.

## How the demo works

This demo styles the Gantt using a prompt with realtime image diffusion. The app adds the Bryntum Gantt to a canvas. When the prompt changes or the Gantt UI updates, the app captures an image of the Gantt and sends it, with the prompt, over a WebSocket to the [fal FLUX.2 \[klein\] realtime image generation and editing API endpoint](https://fal.ai/models/fal-ai/flux-2/klein/realtime). The app uses the [fal client library](https://fal.ai/docs/documentation/model-apis/inference/client-setup) to call models on fal. The model edits the image using the prompt. The example prompts in the app are constructed so that only the style changes:

```
Convert this Bryntum Gantt chart project plan into a hand-stitched felt craft
board with fabric textures and visible stitching. Keep the exact same layout,
task bars, columns and text.
```

The model sends the edited image back, and the app lays it over the Gantt as a click-through overlay. Any further prompt change or Gantt edit triggers another fal API call. Each call sends the current Gantt image and the current prompt, so a prompt-only change re-sends the same image with new instructions.

## Painting the Gantt into a canvas

The app uses the experimental Chromium HTML-in-Canvas API to capture an image of the Bryntum Gantt without rebuilding its DOM. This experimental API is only available in Chrome Canary 149+ or Brave Stable. It also requires enabling the following flags: `chrome://flags/#canvas-draw-element` and `chrome://flags/#enable-experimental-web-platform-features`. The Canvas Draw Element flag is needed to enable painting, and the Experimental Web Platform Features flag is needed to expose the pointer geometry that lets clicks and drags reach the Bryntum Gantt inside the canvas subtree.

A canvas with the `layoutsubtree` attribute makes the browser aware of the HTML content nested inside the canvas, preparing it to be displayed inside the canvas:

```
<canvas
    ref={captureCanvasRef}
    className="capture-canvas"
    {...{ layoutsubtree : 'true' }}
>
    <div className="gantt-wrap" ref={stageRef}>
        <BryntumGantt ref={gantt} {...ganttProps} />
    </div>
</canvas>
```

Chromium fires its `paint` callback when the Gantt’s appearance changes:

``` js
canvas.onpaint = () => {
    const ganttEl = stageRef.current;
    const ctx = canvas.getContext('2d') as ElementDrawingContext | null;

    if (!ganttEl || !ctx) {
        return;
    }

    ctx.reset();
    ctx.drawElementImage(ganttEl, 0, 0);
};
```

The `drawElementImage` method is used to draw the Gantt component in the canvas.

## Sending a square Gantt frame to the FLUX image editing model

The [FLUX realtime endpoint schema](https://fal.ai/models/fal-ai/flux-2/klein/realtime/api) recommends sending a 704 × 704 JPEG at 50% quality. A separate canvas element is used to compress the rectangular Bryntum Gantt into a square Gantt that’s then converted to an image for sending to the FLUX model.

``` js
const square = document.createElement('canvas');
square.width = square.height = INPUT_SIZE;
const squareCtx = square.getContext('2d')!;
squareCtx.fillStyle = '#ffffff';
squareCtx.fillRect(0, 0, INPUT_SIZE, INPUT_SIZE);
squareCtx.drawImage(liveCanvas, 0, 0, INPUT_SIZE, INPUT_SIZE);

const prompt = getPrompt();
const imageUrl = square.toDataURL('image/jpeg', INPUT_JPEG_QUALITY);
```

Where the `INPUT_SIZE` is `704` and the `INPUT_JPEG_QUALITY` is `0.5`. When the app receives the square edited image, it stretches it into a rectangle. The editable Gantt keeps its normal width.

The FLUX WebSocket request uses the current frame as the image, sends the prompt, sets three inference steps and a fixed seed, which we’ll explain in the How the FLUX image editing works section below:

```
connection.send({
    prompt,
    image_url           : imageUrl,
    sync_mode           : true,
    image_size          : 'square',
    num_inference_steps : 3,
    seed                : 35
});
```

Small labels lost too much detail during resizing and diffusion, so the demo app increases the font size and Gantt row height to give them more source pixels. This improves label legibility, although an image model can still alter or misspell text.

## Prompting the style edits

There are 17 premade prompts below the Gantt for editing its style. Each premade prompt asks the model to change the style but retain the structure of the Gantt.

A Bryntum widget text input field at the bottom of the app lets you add a custom prompt. The app sends the typed text as-is, so the user needs to add the layout-preserving wording:

## How the FLUX image editing works

Diffusion models are the most common technique for AI image generation and editing, though FLUX.2 [klein] is technically a [flow model](https://bfl.ai/blog/flux2-klein-towards-interactive-visual-intelligence), which is similar to a diffusion model. It’s a single model for both generation and editing.

When the app sends a request to the FLUX model, the model starts from random noise in a compressed latent representation (a compact grid of numbers) of an image and removes that noise over multiple steps. The three inference steps gradually denoise the representation, using the Gantt image and the prompt at every step, so the final result resembles the Gantt image with the modifications described in the prompt.

Training is what makes the walk from noise to image possible. The model was trained on many images blended with noise at every ratio, learning to predict, for any noisy grid, which direction points toward a real image. Each inference step applies that prediction and moves the grid a little closer to a plausible image; the Gantt frame and the prompt steer which image it moves toward.

The `num_inference_steps` value sets how many of those denoising steps run. More steps generally means a cleaner image, with diminishing returns. We used the default of 3, which gave good results. The maximum number of steps for this model is 8.

The `seed` initializes the pseudorandom generator that produces the starting noise. Its value is a label rather than a dial: no seed is better than another, but the same seed, image, and prompt always return the same output. Pinning `seed : 35` gives every frame the same starting noise, so differences between generated frames come only from real Gantt or prompt changes.

The endpoint also accepts an `output_feedback_strength` value that blends the previous result into that starting point. The schema describes `0.9` as 90% noise plus 10% of the previous output’s latent, which smooths frame-to-frame flicker in exchange for reacting more slowly to change. The demo leaves it at its default of `1`, so every request starts from pure seeded noise.

## Passing interactions through the AI overlay

The latest generated image frame covers the Gantt as an absolutely positioned image. Its CSS includes `pointer-events: none`, so clicks, double-clicks, and drags reach the live Gantt underneath:

```
.ai-overlay {
    position: absolute;
    inset: 0;
    width: 100%;
    height: 100%;
    object-fit: fill;
    pointer-events: none;
}
```

When a task bar moves, Bryntum updates its project and repaints the live DOM. Chromium updates the capture canvas, the next JPEG differs from the previous one, and a later FLUX result replaces the overlay.

## Reusing one WebSocket for changed frames

The browser needs fal credentials to open the realtime connection used by the fal client library. The Vite development server in the demo exposes a dev-only `/api/fal/token` route that uses the fal API key to request a restricted 120-second JSON Web Token (JWT) from fal, and returns that temporary token to the fal client.

``` js
import { fal } from '@fal-ai/client';

...

async function fetchRealtimeToken() : Promise<string> {
    const response = await fetch('/api/fal/token', { method : 'POST' });
    const text = await response.text();
    if (!response.ok) {
        throw new Error(text);
    }
    return text;
}
```

The API route is in a Vite `configureServer` middleware. A production build is static and does not run that hook, so a deployed version needs an authenticated backend or serverless route.

The client then opens one authenticated WebSocket to the realtime fal FLUX runner. Gantt image and prompt inputs and FLUX image editing results are sent as messages on that connection.

```
connection ??= fal.realtime.connect<GanttDiffusionInput>(
    'fal-ai/flux-2/klein/realtime',
    {
        connectionKey          : 'gantt-diffusion',
        throttleInterval       : 128,
        tokenProvider          : fetchRealtimeToken,
        tokenExpirationSeconds : 120,
        onResult               : handleResult,
        onError                : error => onError(error.message ?? String(error))
    }
);
```

The Bryntum Gantt calls the `toggleRunning` function when the ‘Start AI’ button is clicked.

```
{
    type      : 'button',
    ref       : 'startButton',
    text      : 'Start AI',
    rendition : 'filled',
    color     : 'b-purple',
    onClick   : toggleRunning
},
```

This function calls the `createGanttDiffusion` function, which wires the capture canvas and current prompt to the FLUX WebSocket connection and returns a controller with `start`, `stop`, and `nudge` methods:

``` js
const toggleRunning = useCallback(() => {
    diffusionRef.current ??= createGanttDiffusion({
        captureSource : () => captureCanvasRef.current,
        getPrompt     : () => promptRef.current,
        onFrame       : url => {
            setFrameUrl(url);
            setError(null);
        },
        onError : message => setError(message)
    });
    ...
```

Inside `createGanttDiffusion`, a `pump` function drives the loop. Each run captures the current Gantt image from the canvas and compares the image and prompt pair against the last pair sent. If either changed, `pump` sends a fal WebSocket message with the current image and prompt; if nothing changed, it checks again 150 ms later. Every returned frame triggers the next `pump`, and a prompt change calls `nudge` to run it immediately, so the loop keeps offering fresh frames for as long as the AI is running.

## Turning a reference image into prompt text

The prompt input also accepts a style reference image. The browser resizes the uploaded image, which is then sent to the fal `any-llm/vision` endpoint that uses Gemini 2.5 Flash Lite to describe the image style. The returned phrase describes material, texture, lighting, and color palette and is added to the text input.

The text is then sent with the Gantt image to FLUX to alter the style.

## Performance and cost

The styled image overlay adds a cosmetic layer, and most of the visible delay comes from the fal image model round trip. Chromium triggers `canvas.onpaint` when the Gantt’s rendered appearance changes, keeping the canvas up to date during interactions such as task drags. Separately, the code’s `pump` loop copies the current canvas image and encodes it as a 704 × 704 JPEG to check for changes and prepare the next AI request. The canvas can repaint many times between requests, and the underlying Gantt stays interactive while the app waits for each new AI-generated overlay.

The round-trip time for fal requests limits how quickly the overlay updates. Capture, upload, inference, and download took about 1.8 to 4 seconds per frame, which works out to one styled frame every one to two seconds with two requests in flight. These informal measurements vary with the network and fal’s load. [Black Forest Labs quotes sub-second inference for FLUX.2 \[klein\]](https://bfl.ai/blog/flux2-klein-towards-interactive-visual-intelligence), which excludes the browser-to-fal round trip.

Two inference steps blurred the labels, so we kept three. In our tests, keeping two requests in flight improved throughput by overlapping upload with inference. A third request increased the overlay’s lag without improving throughput.

At the quoted rate of $0.00194 per compute second for [fal’s realtime endpoint](https://fal.ai/models/fal-ai/flux-2/klein/realtime), an hour of continuous compute costs about $7. A $5 credit balance would cover about 43 minutes of compute. The `pump` loop skips requests when neither the Gantt image nor the prompt has changed, and the Stop AI button ends the loop.

## Building this demo with the Bryntum MCP server and skills

We built and verified the Bryntum side of this fun, impractical demo using Bryntum’s AI tooling. The [Bryntum MCP server](https://bryntum.com/products/gantt/docs/guide/Gantt/ai-features/mcp-server) gives coding agents version-specific Bryntum documentation. The agent used it to create the Gantt from the [React quick start guide](https://bryntum.com/products/gantt/docs/guide/Gantt/quick-start/react), and to add the [Bryntum widgets](https://bryntum.com/products/gantt/docs/api/widgets) that are used for the demo’s toolbar, buttons, and inputs.

Run the following command in your terminal to add the MCP server to Claude Code:

```
claude mcp add --transport http bryntum https://mcp.bryntum.com
```

Alongside its `search_bryntum_docs` tool, the server exposes guideline resources covering setup, CSS and theme imports, and framework integration, which an agent can read before it writes anything.

The [Bryntum AI Agent skills](https://bryntum.com/products/gantt/docs/guide/Gantt/ai-features/skills) complement the documentation search with practical knowledge for using Bryntum, including the React wrapper patterns and StrictMode handling this app depends on.

## Build it with Bryntum Gantt

Start a free trial, explore live demos, or read the docs.