Local AI pixel-art assistant for Aseprite. Generate sprites from a text prompt, edit existing sprites with an instruction, or redraw a selected region — all running on your own GPU. No cloud, no subscription, your pixels never leave your machine.
Sample output and how it works → vkarach.github.io/spriteloom
| |
Generate · Edit with AI · Inpaint
Rotate / Instruct, two different turnarounds
All clips sped up for the demo.
Real output, one generation each. Subjects like "tall man in a dark coat", "weathered explorer in a red leather coat", "wooden cottage, thatched roof". Scaled up with hard pixel edges for display.
A WebSocket server that runs a diffusion model locally, plus an Aseprite extension that talks to it. You stay in Aseprite the whole time; results open in a side window and drop in as new layers on click. It never edits your existing pixels.
Four tasks, one panel:
Generate— a sprite from a text prompt.** Edit with AI**— change an existing sprite by instruction ("make the sword glow blue"); style and everything unmentioned stays put.** Inpaint Selection**— same, but only ever touches the selected region.** Rotate + Instruct**— re-view the same subject from another angle.
Everything runs on a single model (FLUX.2 Klein), so there are no model swaps: after the first load, tasks respond in seconds.
This runs a 4B-parameter diffusion model on your own machine. It is not a lightweight tool:
| Requirement | |
|---|---|
| GPU | NVIDIA, 12+ GB VRAM (developed on an RTX 5080); 8 GB GPUs work too, in Legacy 8 GB mode — see below |
| RAM | 16 GB minimum. Legacy 8 GB mode holds the whole ~16 GB model in system RAM, so free RAM plus the Windows page file must comfortably clear that — see below |
| OS | Windows |
| Python | 3.11+ |
| Aseprite | 1.3+ |
| Disk | ~15 GB for the model (downloaded during Setup) |
No NVIDIA GPU, no Spriteloom. There is no CPU fallback and no cloud option by design — the whole point is that it runs locally.
Setup's VRAM mode dropdown picks how the ~8 GB transformer is fit onto the GPU:
Auto(default) — detects free VRAM and picks bf16 on 12+ GB cards, Legacy 8 GB mode below that. - bf16— the whole model stays resident on the GPU. Fastest; needs 12+ GB free VRAM. - Legacy 8 GB mode— for 8 GB cards (e.g. a laptop RTX 4060). The model doesn't fit on the GPU all at once, so it moves one layer onto the GPU right before that layer runs, then swaps it back out to system RAM. VRAM usage stays low the whole time (one layer resident, not the whole model), and so does GPU utilization: most of the time goes to shuttling weights over PCIe, not to compute, and that is the actual reason this is slow. Output is identical to bf16, just much slower. There is no faster 8 GB path currently. An fp8-quantized mode was tried and measured no better than this, while adding a real quality risk, so it was dropped.Legacy 8 GB mode keeps the whole ~16 GB model in
system RAM(that is where layers are swapped from). On a 16 GB machine that only fits with a generous Windows page file, so leave it enabled and sized (or set it to system-managed). If free RAM plus page file cannot cover the model the load fails; the server now checks this at startup and prints why instead of crashing.
- Download the latest build from
itch.ioand unzip it anywhere. It has
Spriteloom.exe
,server/
, andplugin/
together; nothing else to fetch first. - Windows will likely show a "Windows protected your PC" SmartScreen
warning —
Spriteloom.exe
isn't code-signed, so it has no reputation yet. ClickMore info, then** Run anyway**. - Press
Setup. It shows what is missing: the environment, PyTorch, the dependencies, the plugin, the model (~15 GB), a Start Menu shortcut. - Tick everything and press
Install selected. It builds the.venv
, installs the packages and the plugin, downloads the model, adds a Start Menu shortcut, and prints a live log. Restart Aseprite once the plugin is in.
Start stays disabled until every required piece — including the model — is in place; there is no lazy first-run download.
If you would rather do it by hand:
py -3 -m venv .venv
.venv\Scripts\python -m pip install -r server\requirements.txt
.venv\Scripts\python -m pip install torch --index-url https://download.pytorch.org/whl/cu128
install-plugin.bat
, then restart Aseprite.
With the .venv
set up this way, start-server.bat
starts the server in a console window instead of through the launcher.
To build the exe yourself from a full clone of this repository:
.venv\Scripts\python -m pip install -r launcher\requirements.txt
, then
.venv\Scripts\python -m PyInstaller build.spec --distpath .
-- it builds
straight into the project root as Spriteloom.exe
, about 15 MB: the model and PyTorch stay outside it.
- Run
Spriteloom.exe
, pressSTART and leave the window open. The dot turns green once the model is resident, about 25 seconds after a warm start. Closing the window stops the server. - In Aseprite:
Sprite → Spriteloom...(or press** F1**). Pick a task, fill the fields, press** Run**. Results open in a separate window; click a variant to insert it as a new layer. Generate understands full sentences: pick aView preset, name theSubject("closed book with dark brown leather cover"), add** Extradetails if needed — the panel shows the exact text it will send. Edit / Inpainttake instructions, not a strength slider: say what to change and how much. Inpaint only touches the selection. Rotate / Instruct**: name the subject explicitly ("four-legged brown horse", not "character"). Optional** Mirror symmetryforces the result left/right symmetric, which can help a front/back view hold together - but it mirrors everything, including a one-handed weapon or an asymmetric pose, so it can also make those look wrong. Try it, don't assume it's always the better result.Advanced... opens a separate window with Background, Palette, Seed, and Extra (appended to the prompt for Generate and Rotate/Instruct; Edit and Inpaint ignore it).Background: Auto detects and strips a uniform background, Remove strips the dominant border color, Keep leaves it fully opaque. Palette**: Auto derives colors per result; Current palette pins output to the open sprite's whole palette; Selected colors pins to only the swatches highlighted in the palette bar; Palette file pins to a.gpl
/.pal
/.png
file so a batch of sprites shares one set of colors.History browses past generations (stored inoutput/
), newest first; click a run to see its variants, click a variant to insert it.
+-----------------------+ +---------------+ +--------------------+
| Aseprite plugin (Lua) | <-- WebSocket --> | Python server | --> | FLUX.2 Klein (GPU) |
| dialogs, results, | | protocol, | | single resident |
| history, layer insert | | postprocess | | model, no swaps |
+-----------------------+ +---------------+ +--------------------+
One model, one warm-up. Every task hits the same FLUX.2 Klein pipeline, so there is no per-task load/unload. On 12+ GB cards it stays fully resident; on 8 GB cards (Legacy 8 GB mode) each layer streams to the GPU per task instead, which is where that mode's extra time goes.WebSocket protocol with request validation at the boundary and streamed progress messages back to the plugin (server/protocol.py
,server/main.py
).Postprocess pipeline turns raw diffusion output into a clean sprite: crop to subject, palette quantization, background removal, mirror symmetry, fit-into-canvas (server/postprocess.py
).The plugin is modular Lua with pure, unit-tested prompt assembly and a UI layer tested against a stubbed Aseprite API — see below.
8-bit quantizing the Klein transformer produces pure noise in text-to-image (edits work fine). The shipped setup sidesteps this: 8-bit text encoder + bf16 transformer, fully resident. See TODO.md.
Server tests:
.venv\Scripts\python -m pytest server/tests/ --ignore=server/tests/smoke.py
Prompt tuning without Aseprite (writes raw + postprocessed variants to output/
):
.venv\Scripts\python -m server.tests.smoke "demonic sword" --size 64
Plugin tests (needs scoop install lua luacheck
, user-scoped):
luacheck plugin\*.lua plugin\tests\*.lua
lua plugin\tests\test_prompt.lua
lua plugin\tests\test_panel.lua
luacheck
bundles Lua 5.4 (the version Aseprite runs); .luacheckrc
declares the API globals it injects. test_panel.lua
loads every module against a stubbed Aseprite API and repaints the status canvas in each server state — this catches broken cross-module calls without launching the editor. Layout and ghosting still need a real Aseprite.
| file | holds |
|---|---|
main.lua |
|
| entry point, registers the menu command | |
dialogs.lua |
|
| the control panel | |
results.lua |
|
| results window (fresh variants) | |
history.lua |
|
| history list and single-run windows | |
ui.lua |
|
| theme colors, checkerboard, variant grid, prompt preview | |
sprite.lua |
|
| frame/mask export, inserting variants as layers | |
prompt.lua |
|
| prompt assembly and key maps (pure Lua, unit-tested) | |
client.lua |
|
| WebSocket client | |
base64.lua |
|
| base64 codec |
| file | holds |
|---|---|
launcher/app.py |
|
| the window, the JS bridge, window sizing | |
launcher/ui/index.html |
|
| markup and styles, main and setup screens | |
launcher/server_proc.py |
|
| the server subprocess, port probing, health | |
launcher/plugin_install.py |
|
| copying the plugin, version comparison | |
launcher/paths.py |
|
| finding the root, Python, Aseprite and the model | |
launcher/setup_checks.py |
|
| what the setup is missing, detection only | |
launcher/setup_steps.py |
|
| running the install steps in order | |
server/config.py |
|
| the settings, shared by the launcher and the server |
The launcher owns the settings file %APPDATA%\Spriteloom\config.json
: the
port, the VRAM mode, and the setup paths all live there, and every write
merges so one key never erases another. The port defaults to 8765, and
Install stamps it into server.json
next to the plugin, so both ends agree.
The server runs inside a Windows job object that dies with the launcher. That is what keeps a crashed or killed launcher from leaving a server behind holding your VRAM.
If the server says it crashed or never finishes , run the built-in check from the install folder:
.venv\Scripts\python -m server.diagnose
It prints package versions, memory headroom, and whether the model files are
complete, then loads the transformer on its own. TRUNCATED
means the download was cut short — delete the model folder and re-download via Setup. A crash only in the full run, with the isolated load passing, is memory: close other apps or raise the Windows page file. The server runs the same file and memory checks at startup, so this is mainly for a closer look.
Apache 2.0. The model, FLUX.2 Klein 4B, is licensed separately by Black Forest Labs under Apache 2.0 and downloaded at runtime, not redistributed here.