{"slug": "spriteloom-local-ai-pixel-art-generator-plugin-for-aseprite", "title": "Spriteloom – local AI pixel-art generator plugin for Aseprite", "summary": "Spriteloom, a local AI pixel-art generator plugin for Aseprite, has been released, enabling users to generate, edit, and inpaint sprites entirely on their own GPU with no cloud or subscription. The tool runs a 4B-parameter diffusion model (FLUX.2 Klein) locally, requiring an NVIDIA GPU with 12+ GB VRAM (8 GB supported in Legacy mode), 16 GB RAM, Windows, Python 3.11+, and Aseprite 1.3+. Developed by vkarach, the plugin offers four tasks—Generate, Edit with AI, Inpaint Selection, and Rotate + Instruct—and is available for download on itch.io.", "body_md": "**Local AI pixel-art assistant for Aseprite.**\nGenerate sprites from a text prompt, edit existing sprites with an\ninstruction, or redraw a selected region — all running on your own GPU.\nNo cloud, no subscription, your pixels never leave your machine.\n\n[Sample output and how it works → vkarach.github.io/spriteloom](https://vkarach.github.io/spriteloom/)\n\n|\n|\n\n*Generate · Edit with AI · Inpaint*\n\n[\n](/vkarach/spriteloom/blob/main/assets/preview/rotate_front.gif)[\n](/vkarach/spriteloom/blob/main/assets/preview/rotate_back.gif)*Rotate / Instruct, two different turnarounds*\n\n*All clips sped up for the demo.*\n\n*Real output, one generation each. Subjects like \"tall man\nin a dark coat\", \"weathered explorer in a red leather coat\", \"wooden cottage,\nthatched roof\". Scaled up with hard pixel edges for display.*\n\nA WebSocket server that runs a diffusion model locally, plus an Aseprite extension that talks to it. You stay in Aseprite the whole time; results open in a side window and drop in as new layers on click. It never edits your existing pixels.\n\nFour tasks, one panel:\n\n**Generate**— a sprite from a text prompt.** Edit with AI**— change an existing sprite by instruction (\"make the sword glow blue\"); style and everything unmentioned stays put.** Inpaint Selection**— same, but only ever touches the selected region.** Rotate + Instruct**— re-view the same subject from another angle.\n\nEverything runs on a single model (FLUX.2 Klein), so there are no model swaps: after the first load, tasks respond in seconds.\n\nThis runs a 4B-parameter diffusion model on your own machine. It is **not**\na lightweight tool:\n\n| Requirement | |\n|---|---|\n| GPU | NVIDIA, 12+ GB VRAM (developed on an RTX 5080); 8 GB GPUs work too, in Legacy 8 GB mode — see below |\n| RAM | 16 GB minimum. Legacy 8 GB mode holds the whole ~16 GB model in system RAM, so free RAM plus the Windows page file must comfortably clear that — see below |\n| OS | Windows |\n| Python | 3.11+ |\n| Aseprite | 1.3+ |\n| Disk | ~15 GB for the model (downloaded during Setup) |\n\nNo NVIDIA GPU, no Spriteloom. There is no CPU fallback and no cloud option by design — the whole point is that it runs locally.\n\nSetup's **VRAM mode** dropdown picks how the ~8 GB transformer is fit onto\nthe GPU:\n\n-\n**Auto**(default) — detects free VRAM and picks bf16 on 12+ GB cards, Legacy 8 GB mode below that. -\n**bf16**— the whole model stays resident on the GPU. Fastest; needs 12+ GB free VRAM. -\n**Legacy 8 GB mode**— for 8 GB cards (e.g. a laptop RTX 4060). The model doesn't fit on the GPU all at once, so it moves one layer onto the GPU right before that layer runs, then swaps it back out to system RAM. VRAM usage stays low the whole time (one layer resident, not the whole model), and so does GPU utilization: most of the time goes to shuttling weights over PCIe, not to compute, and that is the actual reason this is slow. Output is identical to bf16, just much slower. There is no faster 8 GB path currently. An fp8-quantized mode was tried and measured no better than this, while adding a real quality risk, so it was dropped.Legacy 8 GB mode keeps the whole ~16 GB model in\n\n**system RAM**(that is where layers are swapped from). On a 16 GB machine that only fits with a generous Windows page file, so leave it enabled and sized (or set it to system-managed). If free RAM plus page file cannot cover the model the load fails; the server now checks this at startup and prints why instead of crashing.\n\n- Download the latest build from\n[itch.io](https://vkarach.itch.io/spriteloom)and unzip it anywhere. It has`Spriteloom.exe`\n\n,`server/`\n\n, and`plugin/`\n\ntogether; nothing else to fetch first. - Windows will likely show a \"Windows protected your PC\" SmartScreen\nwarning —\n`Spriteloom.exe`\n\nisn't code-signed, so it has no reputation yet. Click**More info**, then** Run anyway**. - Press\n**Setup**. It shows what is missing: the environment, PyTorch, the dependencies, the plugin, the model (~15 GB), a Start Menu shortcut. - Tick everything and press\n**Install selected**. It builds the`.venv`\n\n, installs the packages and the plugin, downloads the model, adds a Start Menu shortcut, and prints a live log. Restart Aseprite once the plugin is in.\n\n**Start** stays disabled until every required piece — including the model —\nis in place; there is no lazy first-run download.\n\nIf you would rather do it by hand:\n\n`py -3 -m venv .venv`\n\n`.venv\\Scripts\\python -m pip install -r server\\requirements.txt`\n\n`.venv\\Scripts\\python -m pip install torch --index-url https://download.pytorch.org/whl/cu128`\n\n`install-plugin.bat`\n\n, then restart Aseprite.\n\nWith the `.venv`\n\nset up this way, `start-server.bat`\n\nstarts the server in a\nconsole window instead of through the launcher.\n\nTo build the exe yourself from a full clone of this repository:\n`.venv\\Scripts\\python -m pip install -r launcher\\requirements.txt`\n\n, then\n`.venv\\Scripts\\python -m PyInstaller build.spec --distpath .`\n\n-- it builds\nstraight into the project root as `Spriteloom.exe`\n\n, about 15 MB: the model\nand PyTorch stay outside it.\n\n- Run\n`Spriteloom.exe`\n\n, press**START** and leave the window open. The dot turns green once the model is resident, about 25 seconds after a warm start. Closing the window stops the server. - In Aseprite:\n**Sprite → Spriteloom...**(or press** F1**). Pick a task, fill the fields, press** Run**. Results open in a separate window; click a variant to insert it as a new layer. **Generate** understands full sentences: pick a**View** preset, name the**Subject**(\"closed book with dark brown leather cover\"), add** Extra**details if needed — the panel shows the exact text it will send.** Edit / Inpaint**take instructions, not a strength slider: say what to change and how much. Inpaint only touches the selection.** Rotate / Instruct**: name the subject explicitly (\"four-legged brown horse\", not \"character\"). Optional** Mirror symmetry**forces the result left/right symmetric, which can help a front/back view hold together - but it mirrors everything, including a one-handed weapon or an asymmetric pose, so it can also make those look wrong. Try it, don't assume it's always the better result.**Advanced...** opens a separate window with Background, Palette, Seed, and Extra (appended to the prompt for Generate and Rotate/Instruct; Edit and Inpaint ignore it).**Background**: Auto detects and strips a uniform background, Remove strips the dominant border color, Keep leaves it fully opaque.** Palette**: Auto derives colors per result; Current palette pins output to the open sprite's whole palette; Selected colors pins to only the swatches highlighted in the palette bar; Palette file pins to a`.gpl`\n\n/`.pal`\n\n/`.png`\n\nfile so a batch of sprites shares one set of colors.**History** browses past generations (stored in`output/`\n\n), newest first; click a run to see its variants, click a variant to insert it.\n\n```\n+-----------------------+                     +---------------+       +--------------------+\n| Aseprite plugin (Lua) |  <-- WebSocket -->  | Python server |  -->  | FLUX.2 Klein (GPU) |\n|   dialogs, results,   |                     |   protocol,   |       |  single resident   |\n| history, layer insert |                     |  postprocess  |       |  model, no swaps   |\n+-----------------------+                     +---------------+       +--------------------+\n```\n\n**One model, one warm-up.** Every task hits the same FLUX.2 Klein pipeline, so there is no per-task load/unload. On 12+ GB cards it stays fully resident; on 8 GB cards (Legacy 8 GB mode) each layer streams to the GPU per task instead, which is where that mode's extra time goes.**WebSocket protocol** with request validation at the boundary and streamed progress messages back to the plugin (`server/protocol.py`\n\n,`server/main.py`\n\n).**Postprocess pipeline** turns raw diffusion output into a clean sprite: crop to subject, palette quantization, background removal, mirror symmetry, fit-into-canvas (`server/postprocess.py`\n\n).**The plugin is modular Lua** with pure, unit-tested prompt assembly and a UI layer tested against a stubbed Aseprite API — see below.\n\n8-bit quantizing the Klein *transformer* produces pure noise in\ntext-to-image (edits work fine). The shipped setup sidesteps this: 8-bit\ntext encoder + bf16 transformer, fully resident. See [TODO.md](/vkarach/spriteloom/blob/main/TODO.md).\n\n**Server tests:**\n\n```\n.venv\\Scripts\\python -m pytest server/tests/ --ignore=server/tests/smoke.py\n```\n\n**Prompt tuning without Aseprite** (writes raw + postprocessed variants to `output/`\n\n):\n\n```\n.venv\\Scripts\\python -m server.tests.smoke \"demonic sword\" --size 64\n```\n\n**Plugin tests** (needs `scoop install lua luacheck`\n\n, user-scoped):\n\n```\nluacheck plugin\\*.lua plugin\\tests\\*.lua\nlua plugin\\tests\\test_prompt.lua\nlua plugin\\tests\\test_panel.lua\n```\n\n`luacheck`\n\nbundles Lua 5.4 (the version Aseprite runs); `.luacheckrc`\n\ndeclares the API globals it injects. `test_panel.lua`\n\nloads every module\nagainst a stubbed Aseprite API and repaints the status canvas in each server\nstate — this catches broken cross-module calls without launching the editor.\nLayout and ghosting still need a real Aseprite.\n\n| file | holds |\n|---|---|\n`main.lua` |\nentry point, registers the menu command |\n`dialogs.lua` |\nthe control panel |\n`results.lua` |\nresults window (fresh variants) |\n`history.lua` |\nhistory list and single-run windows |\n`ui.lua` |\ntheme colors, checkerboard, variant grid, prompt preview |\n`sprite.lua` |\nframe/mask export, inserting variants as layers |\n`prompt.lua` |\nprompt assembly and key maps (pure Lua, unit-tested) |\n`client.lua` |\nWebSocket client |\n`base64.lua` |\nbase64 codec |\n\n| file | holds |\n|---|---|\n`launcher/app.py` |\nthe window, the JS bridge, window sizing |\n`launcher/ui/index.html` |\nmarkup and styles, main and setup screens |\n`launcher/server_proc.py` |\nthe server subprocess, port probing, health |\n`launcher/plugin_install.py` |\ncopying the plugin, version comparison |\n`launcher/paths.py` |\nfinding the root, Python, Aseprite and the model |\n`launcher/setup_checks.py` |\nwhat the setup is missing, detection only |\n`launcher/setup_steps.py` |\nrunning the install steps in order |\n`server/config.py` |\nthe settings, shared by the launcher and the server |\n\nThe launcher owns the settings file `%APPDATA%\\Spriteloom\\config.json`\n\n: the\nport, the VRAM mode, and the setup paths all live there, and every write\nmerges so one key never erases another. The port defaults to 8765, and\n**Install** stamps it into `server.json`\n\nnext to the plugin, so both ends\nagree.\n\nThe server runs inside a Windows job object that dies with the launcher. That is what keeps a crashed or killed launcher from leaving a server behind holding your VRAM.\n\nIf the server says it crashed or never finishes loading, run the built-in loader check from the install folder:\n\n```\n.venv\\Scripts\\python -m server.diagnose\n```\n\nIt prints package versions, memory headroom, and whether the model files are\ncomplete, then loads the transformer on its own. `TRUNCATED`\n\nmeans the\ndownload was cut short — delete the model folder and re-download via Setup. A\ncrash only in the full run, with the isolated load passing, is memory: close\nother apps or raise the Windows page file. The server runs the same file and\nmemory checks at startup, so this is mainly for a closer look.\n\n[Apache 2.0](/vkarach/spriteloom/blob/main/LICENSE). The model, FLUX.2 Klein 4B, is licensed separately by\nBlack Forest Labs under Apache 2.0 and downloaded at runtime, not redistributed\nhere.", "url": "https://wpnews.pro/news/spriteloom-local-ai-pixel-art-generator-plugin-for-aseprite", "canonical_source": "https://github.com/vkarach/spriteloom", "published_at": "2026-08-03 12:06:49+00:00", "updated_at": "2026-08-03 12:25:05.206747+00:00", "lang": "en", "topics": ["generative-ai", "ai-tools", "ai-products", "ai-infrastructure"], "entities": ["Spriteloom", "Aseprite", "FLUX.2 Klein", "NVIDIA", "RTX 5080", "RTX 4060", "vkarach", "itch.io"], "alternates": {"html": "https://wpnews.pro/news/spriteloom-local-ai-pixel-art-generator-plugin-for-aseprite", "markdown": "https://wpnews.pro/news/spriteloom-local-ai-pixel-art-generator-plugin-for-aseprite.md", "text": "https://wpnews.pro/news/spriteloom-local-ai-pixel-art-generator-plugin-for-aseprite.txt", "jsonld": "https://wpnews.pro/news/spriteloom-local-ai-pixel-art-generator-plugin-for-aseprite.jsonld"}}