# Show HN: Local DirectML AI and Floating Toolstrip for PhotoCraft in Rust

> Source: <https://github.com/kobaltgit/photocraft-companion>
> Published: 2026-10-21 07:46:11+00:00

**Extensible plugin ecosystem, local neural tools, and interactive overlay palette for the PhotoCraft graphics editor.**

[Why Companion?](#-why-companion-advantages-over-built-in-photocraft-plugins) • [Features](#-key-features) • [Download & Installers](#-download--installers) • [First-Launch Guide](#-first-launch--onboarding-guide) • [Marketplace](#-community-plugins--marketplace) • [Quick Start](#-quick-start-for-developers) • [Architecture](#-workspace-architecture) • [.pcpl Packages](#-pcpl-package-format) • [Documentation](#-documentation) • [Discord](#-community--discord) • [License](#-license)

Tip

💬 **Join the PhotoCraft Companion Discord:** Have questions about plugins, ideas for new tools, or need developer assistance? **[Join our Discord Server](https://discord.gg/2gzn4Un4Bw)** to connect directly with the author and community developers!

**PhotoCraft Plugins & Companion** solves the core challenge of seamlessly extending the **PhotoCraft** graphics editor (a modern, open-source Rust-based reimplementation of Photoshop):

- **In-Place Canvas Execution:** Neural background removal, color filters, and custom macro actions execute directly on the active layer of your open canvas over JSON-RPC socket (port 18492) and the system clipboard — no manual saving, reopening, or file dialog gymnastics required.
- **Bypassing Sandbox Limitations:** PhotoCraft natively confines internal plugins inside a strict WebAssembly sandbox without local GPU access or filesystem permissions. The companion provides native, hardware-accelerated DirectML GPU inference and local model storage.

PhotoCraft includes a built-in plugin mechanism based on an internal WebAssembly (Wasm) runtime. While Wasm provides sandboxed safety, it imposes strict technical boundaries that make heavy AI workflows and floating palettes impractical.

Here is how **PhotoCraft Companion** elevates the editing experience:

| Feature | Built-in PhotoCraft Wasm Plugins | PhotoCraft Companion & Ecosystem | 
|---|---|---|
| **Local AI & Neural Models** | ❌ **No GPU access** (isolated within Wasm sandbox; cannot run DirectML/Tensor cores) | ✅ **Full DirectML GPU Acceleration** (DirectX 12 / Metal via ONNX Runtime; ~2s inference) | 
| **Model Storage & Autonomy** | ❌ Strict memory limits; cannot package or download multi-hundred MB model weights | ✅ **Autonomous Storage:** Models live inside`plugins/<id>/models/` with SHA-256 validation & in-app downloader | 
| **No-Code Macro Authoring** | ❌ Requires Rust/C toolchains and manual WebAssembly compilation to `.wasm` | ✅ **Zero-Code JSON:** Just list desired PhotoCraft engine commands in`plugin.json` — ready in 60 seconds | 
| **Floating Palette (Toolstrip)** | ❌ Locked inside classic menu bars ( `Filter › Plugins` ); cannot render floating windows | ✅ **Two-Tier Floating Toolstrip:** Compact`egui` palette hovering over the canvas with orientation toggle & tray minimize | 
| **In-App Plugin Marketplace** | ❌ Manual file manipulation into application directories | ✅ **1-Click Marketplace:** Browse online catalog, install, and hot-reload plugins directly from the UI without restarts | 
| **In-Place Pixel Pipeline** |  | ✅ **Zero-Drift In-Place Execution:** Seamless round-trip active layer processing via JSON-RPC socket and clipboard | 

- 
✂️ **In-Place AI Tools & Neural Computer Vision (Zero-Core Architecture):**
  - **AI Remove Background:** Bria AI**RMBG-1.4** (~168 MB) high-precision subject cutout with coordinate preservation up to 8K.
  - **AI Generative Fill (LaMa):** Context-aware zero-prompt object removal and texture inpainting with lasso contour synthesis.
  - **AI Super-Resolution (2x):** Real-ESRGAN Compact with seam-blended tiling to upscale arbitrarily large canvases without VRAM overflow.
  - **Resilient AI Runtime:** Hardware-accelerated**DirectML (DirectX 12 GPU)** with automatic transparent fallback to CPU on unsupported operators.
  - **Ultra-Fast Cached GPU Tiling:** Single-session GPU pipeline caching reducing large canvas inference times up to 25x.
  - **Zero-Core Standard:** Zero plugin-specific logic hardcoded into the companion core; every workflow is strictly declarative via`plugin.json` .
- 
📐 **Photoshop-Style Tool Variations (Sub-Actions & Flyouts):**
  - Tools can declare multiple sub-actions (e.g. Cutout to new layer vs Non-destructive Layer Mask).
  - Buttons render a subtle vector corner chevron; long-pressing (>350ms) or clicking the chevron opens a dedicated floating flyout window (`egui::ViewportBuilder` ) with sub-tools.
  - Active sub-action selection is remembered in persistent user settings.
- 
⚡ **Declarative Workflow Engine & Context Variables:**
  - Multi-step pipelines with variable interpolation (`$source_layer` ,`$result_layer` ,`$active_layer` ,`$doc_width` ,`$doc_height` ,`$has_selection` ,`$fg_color` ,`$bg_color` , and string interpolation`{{...}}` ).
  - Conditional branching (`when: "$has_selection == true"` ), layer variable saving (`save_as` ), error tolerance (`ignore_errors` ), and sandboxed Rhai scripting.
- Multi-step pipelines with variable interpolation (
- 
🔄 **Bidirectional Session Color Sync & Echo Guard:**
  - Two-way real-time color syncing between PhotoCraft tools (Eyedropper `I` , default`D` , swap`X` ) and interactive plugins.
  - Zero-waste polling (180ms only when windows are open) with smart echo prevention (`echo_guard` ).
- Two-way real-time color syncing between PhotoCraft tools (Eyedropper 
- 
🎛️ **Interactive Multi-Viewport UI:**
  - Dedicated floating windows with customizable reactive widgets (sliders, color swatches, hue wheel, contrast diagnostics, dropdowns, toggles) running on Rhai scripting.
- 
🏪 **In-App Community Plugin Marketplace:**
  - Integrated 4th tab inside the Plugin Manager to discover and install community plugins directly from GitHub Releases.
  - Instant 1-click installation of lightweight `.pcpl` archives (< 1 KB) with reactive toolbar hot-reloading (no restart required).
  - Background SVG icon rendering with dark and light theme adaptation.
  - Local offline registry caching for uninterrupted operation.
- 
🎛️ **Floating Two-Tier Toolstrip (`egui` / `eframe`):**
  - Ultra-compact floating palette matching PhotoCraft's native UI aesthetics with pure vector SVG icons.
  - Orientation toggle: seamlessly switch between Horizontal (78 px height) and Vertical (50 px width) modes with zero button clipping.
  - Non-intrusive floating palette behavior: excluded from OS taskbar (`.with_taskbar(false)` ), persistent drag & drop screen coordinates.
  - Floating mini-HUD status badge with live timer and busy lock.
- 
⚙️ **Comprehensive Plugin Manager:**
  - Tabbed interface: `[ General ]` ,`[ Plugins ]` ,`[ Store ]` ,`[ About ]` .
  - Live connection indicator: 3-state real-time socket status (`● Connected` /`● Auth Required` /`○ Offline` ) with token browse dialog (`rfd` ).
  - Native themes: instant synchronization between Dark and pure Light modes.
  - In-App self-updater: checks for new GitHub releases, compares SemVer versions, and downloads platform installers with 1 click.
- Tabbed interface: 
- 
📦 **100% Autonomous `.pcpl` Package Ecosystem:**
  - Plugin neural weights are fully self-contained inside each plugin directory (`plugins/<id>/models/` ).
  - Model integrity verification, progress bars with live transfer speed (MB/s), and automatic missing weights downloader directly in the UI.
- Plugin neural weights are fully self-contained inside each plugin directory (
- 
⏳ **Resilient Waiting / Offline Mode:**
  - The companion does not crash if PhotoCraft is not running; it displays a sleek standby window.
  - Background polling every 1.5s: when PhotoCraft starts, the toolbar smoothly appears above the canvas.
- 
🔔 **Instant Reactive System Tray (0 ms):**
  - System tray icon across Windows, Linux, and macOS with zero-latency dispatching and native Win32 dark context menu support.

Ready-to-use binaries are built for every platform as both full **installers** and standalone **portable archives**:

| Platform | Installer (Setup) | Portable Archive | Notes | 
|---|---|---|---|
| **Windows (x64)** | `PhotoCraft-Companion-Setup-Windows-x64.exe` | `PhotoCraft-Companion-v1.2.2-windows-x64-portable.zip` | Inno Setup installer with tray autostart / No-registry portable zip | 
| **macOS (Apple Silicon & Intel)** | `PhotoCraft-Companion-macos.dmg` | `PhotoCraft-Companion-v1.2.2-macos-portable.tar.gz` | Drag-and-drop DMG image for `/Applications` / Standalone`.app` | 
| **Linux (x86_64)** | `PhotoCraft-Companion-v1.2.2-linux-amd64.deb` | `PhotoCraft-Companion-v1.2.2-linux-x86_64-portable.tar.gz` | Debian package with desktop icon & app menu / Standalone tar.gz | 

Tip

All release binaries are published on the **[GitHub Releases](https://github.com/kobaltgit/photocraft-companion/releases)** page.

Note

**macOS Gatekeeper Note:** If macOS displays a warning saying *"The application is damaged and can't be opened"* or blocks unsigned open-source applications, open Terminal and remove the quarantine attribute:

```
xattr -cr "/Applications/PhotoCraft Companion.app"
```

Alternatively, go to **System Settings › Privacy & Security** and select **Open Anyway**.

PhotoCraft Companion communicates with PhotoCraft through a local control socket (port 18492) using a security token (`control.token`). Starting with **v1.0.7**, launching and pairing are completely automated in one click!

1. **Start PhotoCraft Companion** :
  - If PhotoCraft is not running yet, Companion displays a sleek standby window: *"Waiting for PhotoCraft..."* .
2. If PhotoCraft is not running yet, Companion displays a sleek standby window: 
3. **Launch in 1 Click** :
  - Click the **`[ > Launch PhotoCraft ]`** button directly in the waiting window (or in**Settings › General** ).
  - Companion automatically discovers `photocraft.exe` across standard locations (or you can point to it once via`[ Browse... ]` ), generates a cryptographically secure 64-character`control.token` , launches PhotoCraft with`--control 18492 --control-token-file` , and establishes the connection.
  - The configured executable path is safely persisted and mirrored in portable mode.
4. Click the 
5. **Start Creating** :
  - Once PhotoCraft is running, the waiting window seamlessly transforms into the floating toolbar above your canvas.

If you prefer starting PhotoCraft from custom shortcuts or command line:

1. Launch PhotoCraft with the control socket arguments:

```
photocraft.exe --control 18492 --control-token-file "PhotoCraftData/control.token"
```

2. Start `photocraft-companion.exe` . It automatically detects the running process, reads`control.token` , and turns the connection indicator to**● Connected** .

- 
**Windows:**

```
.\scripts\package_release.ps1
```

 *(Creates a portable `.zip` in `dist/` and builds the Inno Setup installer if ISCC is installed).*
- 
**Linux / macOS:**

```
chmod +x ./scripts/package_release.sh
./scripts/package_release.sh
```

- **OS:** Windows 10 / 11 (x64), Linux (X11 / Wayland), or macOS
- **Rust:** Toolchain 1.80+ (2021 edition)
- **GPU (optional):** Any DirectX 12 capable GPU (NVIDIA, AMD, Intel) for DirectML acceleration.

```
git clone https://github.com/kobaltgit/photocraft-companion.git
cd photocraft-companion
```

Model weights are excluded from Git (`.gitignore`) to keep the repository lightweight. Run the download script to fetch RMBG-1.4:

```
.\scripts\fetch_models.ps1
```

*(Alternatively, click `[ ⬇ Download Model ]` inside the Plugin Manager UI on first run).*

```
# Launch the companion application
cargo run --package companion_app
```

To maintain uncompromising reliability and ensure the Zero-Core architecture remains free of hacks, execute our automated test barriers:

Note

**Resource Guard Rule:** Always execute `cargo` commands with the `-j 1` single-thread flag to prevent CPU and RAM starvation on developer machines.

```
# 1. Code Formatting (Zero-tolerance style check)
cargo fmt --all -- --check

# 2. Complete Workspace Test Suite (152 unit & integration tests)
cargo test --workspace -j 1

# 3. Core Architecture Integrity Linter (Strict Zero-Core & anti-sniffing barrier)
cargo test -p ai_engine --test architecture_integrity_test -j 1

# 4. Dev Regulations Linter (5-file SemVer sync, crate isolation, chronicle parity, model isolation)
cargo test -p ai_engine --test dev_regulations_lint_test -j 1

# 5. UI Linters (Strict i18n localization & Vector-First UI / anti-tofu barrier)
cargo test -p overlay_ui --test i18n_lint_test -j 1
cargo test -p overlay_ui --test vector_first_lint_test -j 1

# 6. Ecosystem Plugin & Registry Validator (Validates manifests, vector SVG icons & model links)
cargo run -p plugin_validator -j 1 -- --validate plugins
```

The project is structured as a modular Cargo Workspace with strict separation of concerns:

```
photocraft-plugins/
├── Cargo.toml                      # Root Rust Workspace manifest
├── crates/                         # Independent crates
│   ├── core_contracts/            # Plugin traits, DTOs, Config, i18n, .pcpl installer
│   ├── photocraft_bridge/         # Async PhotoCraft socket client (JSON-RPC 18492)
│   ├── ai_engine/                 # DirectML GPU ONNX runtime inference (RMBG, Real-ESRGAN, LaMa)
│   ├── overlay_ui/                # Floating Toolstrip, In-App Store, Plugin Manager (egui 0.36)
│   ├── plugin_validator/          # Ecosystem CLI: validation, packaging, marketplace generator
│   └── companion_app/             # Main binary (system tray, lifecycle, DI container, workflow engine)
├── plugins/                       # Installed plugin directory
│   ├── black_and_white/          # Declarative No-Code macro plugin
│   ├── chromasense/              # Interactive UI A11y color panel (Rhai, WCAG 2.1, CVD simulation)
│   ├── depth_map/                 # Autonomous AI 3D depth estimation plugin (Depth Anything V2)
│   ├── inpaint/                   # Autonomous AI generative fill plugin (LaMa)
│   ├── remove_bg/                 # Autonomous AI background removal plugin (RMBG-1.4, Sub-Actions)
│   ├── upscale/                   # Autonomous AI 2x super-resolution plugin (Real-ESRGAN)
│   └── vectorize/                 # Autonomous Smart Vectorizer / SVG Trace algorithm plugin
├── scripts/
│   ├── fetch_models.ps1           # PowerShell model downloader
│   ├── package_release.ps1        # Inno Setup & Windows packaging script
│   └── package_release.sh         # Linux DEB & macOS DMG packaging script
└── docs/                          # Engineering specifications & dev chronicles
```

Any plugin can be distributed as a portable `.pcpl` archive (ZIP format):

```
my_plugin.pcpl
├── plugin.json              # Declarative manifest (metadata, commands, hotkeys)
├── icon.svg                 # Vector palette icon
└── models/                  # (Optional) Local ONNX neural weights
    └── rmbg-1.4.onnx
```

Install any `.pcpl` file with a single click via `[ + Install Plugin (.pcpl) ]` inside the Plugin Manager.

Download community plugins as ready-to-use `.pcpl` packages and drop them into your Companion or install them in 1-click via the **Plugin Manager**:

| Icon | Plugin & Description | Type | Version | Author | Package (.pcpl) | 
|---|---|---|---|---|---|
|  | **B&W Film Pop (No-Code)*** Duplicates active layer, converts to grayscale with auto-contrast adjustment.* | `⚡ Macro` | `1.0.0` | kobaltGIT | [📥 Download .pcpl](https://github.com/kobaltgit/photocraft-companion/releases/download/plugins/black_and_white.pcpl) | 
|  | **ChromaSense*** Accessibility Color Assistant for artists with color vision deficiencies* | `🎨 UI Panel` | `1.0.0` | kobaltGIT | [📥 Download .pcpl](https://github.com/kobaltgit/photocraft-companion/releases/download/plugins/chromasense.pcpl) | 
|  | **AI Depth Map (Depth Anything V2)*** State-of-the-art monocular 3D depth estimation for Lens Blur, bokeh and atmospheric lighting via Depth Anything V2.* | `🤖 AI (ONNX)` | `1.0.0` | kobaltGIT | [📥 Download .pcpl](https://github.com/kobaltgit/photocraft-companion/releases/download/plugins/depth_map.pcpl) | 
|  | **AI Generative Fill (LaMa)*** Zero-prompt neural generative fill and object eraser for PhotoCraft using LaMa model via DirectML GPU.* | `🤖 AI (ONNX)` | `1.0.0` | kobaltGIT | [📥 Download .pcpl](https://github.com/kobaltgit/photocraft-companion/releases/download/plugins/inpaint.pcpl) | 
|  | **AI Remove Background (RMBG-1.4)*** High-fidelity neural background removal for PhotoCraft using Bria AI RMBG-1.4 model via local ONNX Runtime.* | `🤖 AI (ONNX)` | `1.1.0` | kobaltGIT | [📥 Download .pcpl](https://github.com/kobaltgit/photocraft-companion/releases/download/plugins/remove_bg.pcpl) | 
|  | **AI Super-Resolution (2x)*** 2x neural super-resolution upscale for PhotoCraft active layer via local ONNX Runtime & DirectML GPU.* | `🤖 AI (ONNX)` | `1.0.0` | kobaltGIT | [📥 Download .pcpl](https://github.com/kobaltgit/photocraft-companion/releases/download/plugins/upscale.pcpl) | 
|  | **Smart Vectorizer / SVG Trace*** High-fidelity vectorization and SVG tracing of raster layers with Bezier curve fitting.* | `📐 Algorithm` | `1.0.0` | kobaltGIT | [📥 Download .pcpl](https://github.com/kobaltgit/photocraft-companion/releases/download/plugins/vectorize.pcpl) | 

Tip

**Want to publish your own plugin?** Check out the [Plugin Development Guide](https://github.com/kobaltgit/photocraft-companion/blob/main/docs/PLUGIN_DEVELOPMENT_GUIDE.en.md), create your plugin inside `plugins/<id>/`, and submit a Pull Request! Our automated CI validates your manifest, icon, and neural models before publishing.

Detailed specifications and development history can be found in [`docs/`](https://github.com/kobaltgit/photocraft-companion/blob/main/docs):

- 📘 [`GEMINI.md`](https://github.com/kobaltgit/photocraft-companion/blob/main/GEMINI.md) — Master architectural manifest.
- 📋 [`docs/AGENTS_MASTER_GUIDE.md`](https://github.com/kobaltgit/photocraft-companion/blob/main/docs/AGENTS_MASTER_GUIDE.md) — Master development regulations and AI agent guidelines.
- 🎯 [`docs/PLAN.md`](https://github.com/kobaltgit/photocraft-companion/blob/main/docs/PLAN.md) — Implementation blueprint for floating UI and autonomous plugins.
- 🗺️ [`docs/ROADMAP.md`](https://github.com/kobaltgit/photocraft-companion/blob/main/docs/ROADMAP.md) — Project roadmap and milestones.
- ✅ [`docs/CHECKLIST.md`](https://github.com/kobaltgit/photocraft-companion/blob/main/docs/CHECKLIST.md) — Interactive phase-by-phase checklist.
- 🐛 [`docs/BUGS_AND_ISSUES.md`](https://github.com/kobaltgit/photocraft-companion/blob/main/docs/BUGS_AND_ISSUES.md) — Defect tracking log (BUG-001 through BUG-075).
- 📜 [`docs/CHRONICLE.md`](https://github.com/kobaltgit/photocraft-companion/blob/main/docs/CHRONICLE.md) /[`docs/CHRONICLE.en.md`](https://github.com/kobaltgit/photocraft-companion/blob/main/docs/CHRONICLE.en.md) — Chronological developer devlog in Russian and English.
- 🛠️ [`docs/PLUGIN_DEVELOPMENT_GUIDE.en.md`](https://github.com/kobaltgit/photocraft-companion/blob/main/docs/PLUGIN_DEVELOPMENT_GUIDE.en.md) /[`RU`](https://github.com/kobaltgit/photocraft-companion/blob/main/docs/PLUGIN_DEVELOPMENT_GUIDE.md) — Plugin Developer Guide (No-Code macros, AI inference, workflows).
- 📖 [`docs/PHOTOCRAFT_COMMANDS_REFERENCE.en.md`](https://github.com/kobaltgit/photocraft-companion/blob/main/docs/PHOTOCRAFT_COMMANDS_REFERENCE.en.md) /[`RU`](https://github.com/kobaltgit/photocraft-companion/blob/main/docs/PHOTOCRAFT_COMMANDS_REFERENCE.md) — Complete Catalog of all**851** PhotoCraft Engine Commands.
- 📐 [`docs/UNIVERSAL_PLUGIN_CORE_SPECIFICATION.md`](https://github.com/kobaltgit/photocraft-companion/blob/main/docs/UNIVERSAL_PLUGIN_CORE_SPECIFICATION.md) — Universal Core specification (Rhai, CLI runner, C-ABI dynamic libs).

Have questions, suggestions, or want to share your custom plugins with fellow digital artists?
Join the official **PhotoCraft Companion Discord Server**:

**kobaltGIT** ([GitHub](https://github.com/kobaltgit))

This project is licensed under the [MIT License](https://github.com/kobaltgit/photocraft-companion/blob/main/LICENSE).